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Introduction 


Ina V.S. Mullis 


PIRLS 2016— Monitoring Trends in Reading 
Literacy Achievement 

Reading literacy is one of the most important abilities students acquire as they 
progress through their early school years. It is the foundation for future learning 
across all subjects as well as for personal growth and recreation (National 
Institute of Child Health and Human Development, 2000; Stanovich, 1986). 
Reading literacy also equips young students with the foundational skills that 
will be needed in order to participate fully in their communities and the larger 
society (Organisation for Economic Cooperation and Development, 2010). 

Because developing reading literacy ability is vital to every student’s 
growth and maturation, the International Association for the Evaluation of 
Educational Achievement, more widely known as IEA, has been conducting 
regular international assessments of reading literacy and the factors associated 
with its acquisition in countries around the world for more than 50 years. 

IEA is an independent international cooperative of national research 
institutions and government agencies that has been conducting studies of cross- 
national achievement since 1959. IEA pioneered international comparative 
assessment of educational achievement in the 1960s to gain a deeper 
understanding of policy effects across countries’ different systems of education. 


The PIRLS, prePIRLS, and ePIRLS International 
Assessments 

PIRLS (Progress in International Reading Literacy Study) was inaugurated in 
2001 as a follow-up to IEA’s 1991 Reading Literacy Study. Conducted at five- 
year intervals, PIRLS 2016 is the fourth assessment in the current trend series, 
following PIRLS 2001, 2006, and 2011. All of the countries, institutions, and 
agencies involved in successive PIRLS assessments have worked collaboratively 
in building the most comprehensive and innovative measure of reading 
comprehension possible, beginning in 2001 and improving with each cycle since 
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then. PIRLS is directed by the TIMSS & PIRLS International Study Center at 
Boston College. 

PIRLS focuses on the achievement of young students in their fourth year 
of schooling and the experiences they have at home and at school in learning 
to read. PIRLS assesses students in their fourth year of schooling because it is 
an important transition point in childrens development as readers. Typically, 
at this point in their schooling, students have learned how to read and are 
now reading to learn. As such, PIRLS assesses how well students can read with 
understanding, and use their reading to gain new insights and information. 

However, the language and reading curriculum varies considerably across 
countries, so as part of PIRLS 2011, IEA extended PIRLS to meet the needs 
of countries where most children in the fourth grade are still developing 
fundamental reading skills. For example, if students are more likely to have 
developed the reading comprehension competencies necessary for success on 
PIRLS by the fifth or sixth grade, IEA encourages participation in PIRLS at 
those grades. 

Most importantly, PIRLS was extended in 2011 to include prePIRLS — an 
assessment that reflects the same conception of reading as PIRLS, except it is 
less difficult and is designed to test basic reading skills that are a prerequisite for 
PIRLS. The prePIRLS 2016 reading literacy assessment offers an excellent basis 
for countries with relatively low levels of learning to systematically measure and 
improve children’s learning outcomes. 

New for 2016, ePIRLS is an innovative assessment of online reading. In 
order to be responsive to the information age and provide important information 
about how well students are developing 21 st century skills, ePIRLS uses an 
engaging, simulated Internet environment to present fourth grade students with 
authentic school-like assignments involving science and social studies topics. A 
teacher avatar guides students through the ePIRLS assignments, prompting the 
students with questions about the online information. 

The number of countries participating in PIRLS has grown with each 
assessment cycle. Approximately 60 countries from around the world are 
planning to participate in PIRLS 2016. In 2011, nationally representative samples 
of students in 50 countries participated in the various assessment options 
included in PIRLS. In total, 325,000 students participated in PIRLS 2011. 
The results for the PIRLS 2011 assessments were reported in PIRLS 2011 
International Results in Reading (Mullis, Martin, Foy, & Drucker, 2012). This 
report summarized trends in students’ reading literacy achievement using the 
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PIRLS International Benchmarks and presented the results for the first prePIRLS 
assessment. The report also presented a rich array of information about the 
students’ home backgrounds and their attitudes toward reading, school 
contexts for learning to read, teachers’ education and training, and classroom 
characteristics and activities. 

Monitoring Trends 

From its inception, PIRLS was designed to measure trends in reading literacy 
achievement as well as contexts for learning to read in school and at home. 
Conducted every five years, international assessments have been conducted in 
2001, 2006, 2011, and now 2016, with the next assessment planned for 2021. 
Many of the countries participating in PIRLS 2016 also participated in the 
previous study cycles. For countries with data since 2001, PIRLS 2016 will 
provide the fourth in a series of trend measures collected over 15 years. These 
countries will have the opportunity to evaluate progress in reading achievement 
across four time points: 2001, 2006, 2011, and 2016. 

The PIRLS Framework for Assessing 
Reading Literacy 

Based on reading purposes and comprehension processes, the PIRLS framework 
provides the foundation for the PIRLS, prePIRLS, and ePIRLS assessments of 
students’ reading achievement. 1 

PIRLS assesses students’ reading achievement within the two overarching 
purposes for reading that account for most of the reading done by young 
students both in and out of school: 

• Reading for literary experience; and 

• Reading to acquire and use information. 

The PIRLS assessments integrate four types of comprehension processes 
within each of the two purposes for reading: 

• Focus on and retrieve explicitly stated information; 

• Make straightforward inferences; 

• Interpret and integrate ideas and information; and 

• Evaluate and critique content and textual elements. 


1 The purposes for reading and processes of comprehension are fully described in Chapter 1. 
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Policy Relevant Data about the Contexts for 
Learning to Read 

In order to provide an important context for interpreting the reading 
achievement results, PIRLS collects considerable background information 
about how educational opportunities are provided to students as well as the 
factors that influence how students use these opportunities. These background 
data include information about national curriculum policies in reading and 
how the educational system is organized to facilitate learning, students’ home 
environment for learning, school climate and resources, and how instruction 
actually occurs in classrooms (see Chapter 2). 

The PIRLS Encyclopedia has been published with each assessment 
cycle since 2001. Each PIRLS country prepares a chapter summarizing the 
structure of its education system, the language and reading curriculum in 
the primary grades, and overall policies related to reading instruction (e.g., 
teacher education, instructional materials, and assessment). The PIRLS 2011 
Encyclopedia (Mullis, Martin, Minnich, Drucker, & Ragan, 2012) is a valuable 
compendium of information about how reading is taught around the world 
and provides an indispensable resource for policy and research in comparative 
education. 

In order to obtain the background information that is published together 
with the PIRLS achievement results, PIRLS asks students, their parents, their 
teachers, and their school principals to complete questionnaires about their 
home, school, and classroom contexts for learning to read. The questionnaire 
data provide important profiles of teaching and learning across the countries. 
However, reading literacy involves not only the ability to construct meaning 
from a variety of texts, but also behaviors and attitudes that support becoming a 
lifelong reader. Behaviors and attitudes about reading literacy are an important 
part of lifelong reading and contribute to the full realization of the individual’s 
potential within a literate society (Organisation for Economic Cooperation and 
Development, 1995; 1997; 2000; 2005; 2010). 

In particular, the Learning to Read Survey, completed by students’ parents 
and caregivers, has been an important component of each assessment cycle 
of PIRLS since 2001. It provides valuable information about students’ home 
support for early literacy learning and reading experiences. Also, the student 
questionnaire contains a series of questions about students’ behaviors and 
attitudes toward reading literacy. 
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The upcoming PIRLS 2016 assessment with its questionnaires, as well 
as the PIRLS 2016 Encyclopedia, will collect and report data on a variety of 
activities and experiences from the following range of learning to read contexts: 

• National and community; 

• Home; 

• School; 

• Classroom; and 

• Student. 

As a result, the PIRLS 2016 Encyclopedia and questionnaire data will 
provide a dynamic picture of reading educational policies and practices across 
the participating countries that can raise issues and indicate avenues relevant 
to educational improvement efforts. 

Extending PIRLS to Meet the Needs of 
Many Countries 

The fourth year of schooling was chosen as a focal point for PIRLS because by 
this time in their schooling many children have developed into independent 
readers. For a variety of reasons, however, there are countries where most 
children in the fourth grade are still developing fundamental reading literacy 
skills. Thus, beginning in 2011 and continuing in 2016, IEA has extended PIRLS 
to meet the needs of these countries by offering PIRLS at grade levels beyond 
the fourth grade and by developing a less difficult reading assessment designed 
to be a stepping-stone to PIRLS. 

Consistent with the purpose of extending PIRLS or bridging to PIRLS, 
the less difficult reading assessment has been named prePIRLS. The prePIRLS 
assessment follows the same conception of reading literacy as PIRLS outlined in 
this publication, but is intended to measure the reading comprehension skills of 
students who are still in the process of learning how to read. The availability of 
prePIRLS enables IEA to target the PIRLS assessment to each country’s situation 
in order to provide the best possible measurement. 

Depending on a country’s educational development and the students’ 
reading level, countries can participate in either or both PIRLS and prePIRLS 
in order to conduct the most effective assessment. The goal is to provide the 
best policy- relevant information about how to improve teaching and learning 
and to help young students become accomplished and self-sufficient readers. 



TIMSS Er PIRLS 

International Study Center 

Lynch School of Education, Boston College 


PIRLS 2016 FRAMEWORK INTRODUCTION 









PIRLS 



2016 



PIRLS 201 6 at the 5th or 6th Grades 

In some countries, students are more likely to have developed the reading 
comprehension competencies necessary for success on PIRLS later than the 
fourth grade, perhaps by the fifth or sixth grades. In these countries, IEA 
encourages participation in PIRLS 2016 at the fifth or sixth year of schooling. 
Participation in PIRLS 2016 at a later grade is important for such countries 
because it can provide valuable information about students’ strengths and 
weaknesses in learning to read and reading to learn, whereas earlier participation 
could provide little information except that the assessment was too difficult. 

prePIRLS 2016 

Considering increased efforts to raise literacy levels in many countries around 
the world, IEA has developed an assessment specifically for countries where 
many students’ reading skills are emerging, but are not yet fully developed. As 
noted, the prePIRLS 2016 assessment reflects the same conception of reading 
as PIRLS 2016, except it is less difficult and is designed to test basic reading 
skills that are a prerequisite for PIRLS. The reading passages are shorter, with 
easier vocabulary and syntax. Students’ ability to read and answer questions 
about these passages can provide valuable information about their strengths 
and weaknesses in reading comprehension. Several countries participated 
in prePIRLS in 2011, including Colombia, where both PIRLS and prePIRLS 
were administered to fourth grade students. This enabled the TIMSS & PIRLS 
International Study Center to create a link between performance on prePIRLS 
and PIRLS. The prePIRLS assessment has proved to provide important data 
about reading learning levels in the PIRLS context, and offers an excellent 
basis for countries with relatively low levels of reading literacy achievement to 
systematically measure and improve young students’ reading outcomes. 

Introduced with the upcoming assessment cycle, TIMSS 2015 also has 
a new, less difficult mathematics assessment called TIMSS Numeracy. TIMSS 
Numeracy assesses fundamental mathematical knowledge, procedures, and 
problem-solving strategies that are prerequisites for success on TIMSS. TIMSS 
Numeracy asks students to answer questions and work problems similar to 
TIMSS, except with easier numbers and more straightforward procedures. 

Together, prePIRLS and TIMSS Numeracy are intended to be responsive to 
the needs of the global education community and can support efforts that work 
toward universal learning for all. As debates shift from access for all to learning 
for all, both assessments can provide a much-needed means of effectively 
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can help countries and international organizations effectively measure and 
thereby improve reading and mathematics learning outcomes for young 
students worldwide. 

ePIRLS 2016— An Innovative, Online Reading 
Assessment 

New for 2016, ePIRLS is an innovative assessment of online reading. The 
range of information available on the Internet has exploded to the point that, 
for many people, the Internet has become the primary source for obtaining 
information at work, at home, and for school. Particularly relevant to PIRLS 
2016, Internet reading increasingly is becoming one of the central ways students 
are acquiring information (Leu, Kinzer, Coiro, Castek, & Henry, 2013; Leu, 
O’Byrne, Zawilinski, McVerry, & Everett-Cacopardo, 2009; Murnane, Sawhill, 
& Snow, 2012). It is very common for students in many countries to consult the 
Internet to complete school assignments or tasks. The Internet also is becoming 
the central source for students to gather additional information in their school 
subjects, such as science and social studies. As students have begun to rely on 
the Internet, reading curricula around the world are beginning to emphasize 
the importance of developing online reading skills and competencies such as 
reading for information (see PIRLS 2011 Encyclopedia). 

Considering the global emphasis on Internet use, the ePIRLS computer- 
based assessment of online reading is a necessary extension of the PIRLS 2016 
assessment in order to adequately measure students’ reading achievement, 
especially in informational reading. The ePIRLS assessment uses an engaging, 
simulated Internet environment to present fourth grade students with authentic 
school-like assignments involving science and social studies topics. 

An Internet browser window provides students with websites containing 
information about their assignments, and students navigate through pages 
with a variety of features, such as photos, graphics, videos, multiple tabs, 
and hyperlinks. In an assessment window, a teacher avatar guides students 
through the ePIRLS assignments, prompting the students with questions. 
The development of the ePIRLS approach to assessing online reading 
benefited greatly from the work of ORCA (the Online Reading Comprehension 
Assessment), where students research science issues in a web environment (see 
Leu, Kulikowich, Sedansk, & Coiro, 2008). 

In order to successfully complete ePIRLS, students must not only be able 
to navigate and discriminate among informational texts in a non-linear online 
environment, students also must be able to use these online sources to retrieve 
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data, make inferences, and integrate and evaluate the online information 
presented on individual web pages. Importantly, at the end of the online 
assessment task, students must be able to integrate the ideas and information 
presented across multiple web sites. 

Using PIRLS Data for Educational Improvement 

As reported in the PIRLS 2011 Encyclopedia, countries use PIRLS data for 
system-level monitoring of education achievement in a global context. They 
compare their reading achievement levels and contexts for learning with those of 
other countries, and monitor progress in reading achievement over time. Many 
countries reported initiating educational reforms when PIRLS achievement 
results were low compared to other countries, or lower than expected. That 
is, many countries also view the PIRLS results in the context of national goals. 
Working to achieve equity provided another impetus for reform and a number 
of countries reported having made special efforts to reduce achievement 
disparities among ethnic, social, or regional groups. Countries implementing 
educational changes typically look to future PIRLS assessment cycles to monitor 
improvement. 

PIRLS data, framework, released items, and scoring guides often are used 
as a basis for updating curriculum and textbooks, as well as for improving 
classroom instruction, primarily through teacher workshops and training 
programs. Many countries reported increased sponsorship of reading research 
activity, including research using PIRLS data, and several have established 
national reading centers. 

Overview of the Assessment Design 

The PIRLS assessment design and specifications are discussed in detail in 
Chapter 3. In PIRLS and prePIRLS, nationally representative samples of students 
are assessed using test booklets based on a rotated booklet design. Following 
the same procedures as used in prior assessments, the PIRLS 2016 booklets 
contain five literary and five informational passages. The prePIRLS 2016 
booklets contain four literary and four informational passages. The passages 
are distributed across the test booklets, with every booklet comprised of two 
passages and each passage’s accompanying test questions. 

The day following the PIRLS 2016 assessment, ePIRLS will use computers 
to assess the students in online informational reading. ePIRLS consists of four 
authentic online reading tasks, with each student completing two tasks and a 
short questionnaire about familiarity with online reading. 
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As noted earlier, information is gathered about students’ home and 
school experiences in developing reading literacy as well as their attitudes 
toward reading. PIRLS and prePIRLS include an extensive set of background 
questionnaires given to students’ parents, teachers, and school principals. 
Countries complete questionnaires about their education systems and reading 
curricula, and prepare a chapter for the PIRLS 2016 Encyclopedia. 

Updating the PIRLS Framework for the 
PIRLS 201 6 Assessments 

Updating the PIRLS framework with each assessment cycle provides 
participating countries opportunities to introduce fresh ideas and current 
information about curricula, standards, frameworks, and instruction. This keeps 
the frameworks educationally relevant, creates coherence from assessment to 
assessment, and permits the framework, instruments, and procedures to evolve 
gradually into the future. 

For PIRLS 2016, the framework was updated using information provided 
by the National Research Coordinators (NRCs) from the participating countries 
and the PIRLS 2011 Encyclopedia. In addition, the PIRLS 2016 reading expert 
committee (Reading Development Group) provided direction. Using an iterative 
process, the framework, under the direction of the committee, were once again 
reviewed by the NRCs and updated a final time prior to publication. 
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CHAPTER 1 

PIRLS 2016 Reading Framework 

Ina V.S. Mullis, Michael O. Martin, and Marian Sainsbury 


The PIRLS 2016 Reading Framework and the instruments developed to 
assess this framework reflect IEA’s commitment to be forward thinking and 
incorporate the latest approaches to measuring the reading achievement of 
young students in their fourth year of schooling. PIRLS is based on the broad 
notion of what the ability to read means — a notion that includes the ability to 
reflect on written texts and to use these texts as tools for attaining individual 
and societal goals, also known as “reading to do” (Stiggins, 1982). This view is 
increasingly relevant in todays society, where greater emphasis continues to 
be placed on students’ ability to use the information they gain from reading 
(Organisation for Economic Cooperation and Development, 1995; 1997; 
2000; 2001; 2005; 2010). Emphasis is shifting from demonstrating fluency 
and basic comprehension to demonstrating the ability to apply what is read 
to new situations or projects (Coulombe, Trembly, & Marchand, 2004; Smith, 
Mikulecky, Kibby, & Dreher, 2000; see also PIRLS 2011 Encyclopedia). 

The PIRLS framework for assessing reading achievement was initially 
developed for the first assessment in 2001, using IEA’s 1991 Reading Literacy 
Study (Elley, 1992; 1994; Wolf, 1995) as the basis for the PIRLS definition of 
reading literacy and for establishing the aspects of reading comprehension to 
be assessed. Since then, the PIRLS assessment framework has been updated for 
each subsequent assessment cycle (Campbell, Kelly, Mullis, Martin, & Sainsbury, 
2001; Mullis, Kennedy, Martin, & Sainsbury, 2006; Mullis, Martin, Kennedy, 
Trong, & Sainsbury, 2009). 


A Definition of Reading Literacy 

The PIRLS definition of reading literacy is grounded in IEA’s 1991 study, in 
which reading literacy was defined as “the ability to understand and use those 
written language forms required by society and/or valued by the individual.” 
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With successive assessments, this definition has been elaborated so that it retains 
its applicability to readers of all ages and a broad range of written language 
forms, yet makes explicit reference to aspects of the reading experience of 
young students as they become proficient readers, highlights the widespread 
importance of reading in school and everyday life, and acknowledges the 
increasing variety of texts in todays technological world. Currently, the PIRLS 
definition of reading literacy is as follows: 

Reading literacy is the ability to understand and use those 
written language forms required by society and/or valued by the 
individual. Readers can construct meaning from texts in a variety 
of forms. They read to learn, to participate in communities of 
readers in school and everyday life, and for enjoyment. 

This view of reading reflects numerous theories of reading literacy as a 
constructive and interactive process (Anderson & Pearson, 1984; Chall, 1983; 
Kintsch, 1998; 2012; 2013; Ruddell & Unrau, 2004; Rumelhart, 1985). Meaning 
is constructed through the interaction between reader and text in the context 
of a particular reading experience (Britt, Goldman, & Rouet, 2012; Snow, 2002). 
Readers are regarded as actively constructing meaning as well as knowing 
effective reading strategies and how to reflect on reading (Afflerbach & Cho, 
2009; Langer, 2011). 

Before, during, and after reading, readers use a repertoire of linguistic 
skills, cognitive and metacognitive strategies, as well as background knowledge 
to construct meaning (Baker & Beall, 2009; Kintsch, 2012; 2013; Pressley & 
Gaskins, 2006; Rapp & van den Broek, 2005). In addition, the context of the 
reading situation can support the construction of meaning by promoting 
engagement and motivation to read, but the context also can place specific 
demands that might not support the construction of meaning (Christianson & 
Luke, 2011; Lorch, Lemarie, & Grant, 2011; Miller & Fair cloth, 2009; Taboada, 
Tonks, Wigfield, & Guthrie, 2009). 

In order to acquire knowledge of the world and themselves, readers can 
learn from a host of text types. Any given text type can take many forms and 
combinations of forms. These include traditional written forms, such as books, 
magazines, documents, and newspapers as well as digital forms such as email, 
text messaging, and Internet websites where text often is integrated with various 
multimedia formats (Leu, Kinzer, Coiro, & Cammack, 2004; Leu, Kinzer, Coiro, 
Castek, & Henry, 2013; Rosell & Pahl, 2010; Reuda, 2013). 
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Discussing what they have read with different groups of individuals allows 
young students to construct text meaning in a variety of contexts (Almasi & 
Garas-York, 2009; Murphy, Wilkinson, Soter, Hennessey, & Alexander, 2009). 
Social interactions about reading in one or more communities of readers can be 
instrumental in helping young students gain an understanding and appreciation 
of texts (Galda & Beach, 2001; Kucer, 2005). Socially constructed environments 
in the classroom or school library can give young students formal and informal 
opportunities to broaden their perspectives about texts and to see reading as 
a shared experience with their classmates and teachers (Alvermann & Moje, 
2013; Guthrie, 1996). This can be extended to communities outside of school as 
young students talk with their families and friends about ideas and information 
acquired from reading. 

Overview of the PIRLS Framework for Assessing 
Reading Achievement 

Based on reading purposes and comprehension processes, the PIRLS 
framework provides the foundation for the PIRLS assessment of students’ 
reading achievement in their fourth year of schooling, as well as for prePIRLS, 
a literacy assessment that is an easier version of PIRLS, and ePIRLS, which 
extends PIRLS to assess online reading. As shown in Exhibit 1, the PIRLS 
framework focuses on the two overarching purposes for reading that account 
for most of the reading done by young students both in and out of school: for 
literary experience, and to acquire and use information. In addition, the PIRLS 
assessment integrates four broad-based comprehension processes within each of 
the two purposes for reading: focus on and retrieve explicitly stated information, 
make straightforward inferences, interpret and integrate ideas and information, 
and evaluate and critique content and textual elements. 

Exhibit 1 . The PIRLS Reading Purposes and Comprehension 


Purposes for Reading 


Literary Experience 
Acquire and Use Information 


Processes of Comprehension 


Focus on and Retrieve Explicitly Stated Information 
Make Straightforward Inferences 
Interpret and Integrate Ideas and Information 
Evaluate and Critique Content and Textual Elements 
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It should be acknowledged that the purposes for reading and the processes 
of comprehension do not function in isolation from one another or from the 
context in which students live and learn. 

PIRLS Framework Emphases in PIRLS, prePIRLS, 
and ePIRLS 

Although the two reading purposes and four comprehension processes form 
the basis for assessing PIRLS as well as prePIRLS and ePIRLS, there are some 
differences in emphases across the assessments. Exhibit 2 presents the reading 
purposes and processes assessed by PIRLS and the percentages of the test 
devoted to each for PIRLS, prePIRLS, and ePIRLS. 

Exhibit 2: Percentages of the PIRLS, prePIRLS, and ePIRLS Reading 

Assessments Devoted to Each Reading Purpose and 
Comprehension Process 


PIRLS 

prePIRLS 

ePIRLS 

Purposes for Reading 

Literary Experience 

50% 

50% 

0% 

Acquire and Use Information 

50% 

50% 

100% 

Processes of 
Comprehension 

Focus on and Retrieve Explicitly 
Stated Information 

20% 

50% 

20% 

Make Straightforward Inferences 

30% 

25% 

30% 

Interpret and Integrate Ideas and 
Information 

30% 

25% 

30% 

Evaluate and Critique Content and 
Textual Elements 

20% 

20% 


Both PIRLS and prePIRLS devote half of the assessment passages to each 
of the purposes for reading, while the ePIRLS online assessment focuses solely 
on reading to acquire and use information. The ePIRLS approach simulates 
websites from the Internet, through which students can navigate to accomplish 
school-based research projects or tasks. Because prePIRLS is designed for 
students earlier in the process of learning to read, a larger percentage of items 
(50 percent of the assessment) are devoted to measuring foundational reading 
comprehension processes — the ability to focus on and retrieve explicitly stated 
information. Also, prePIRLS has shorter reading passages with easier vocabulary 
and syntax. 


16 


CHAPTER 1 








Purposes for Reading 

Throughout the world, reading literacy is directly related to the reasons people 
read; broadly, these reasons include reading for pleasure and personal interest, 
learning, and participation in society The early reading of most young students 
centers on the first two reasons, and thus often includes reading of narrative 
texts that tell a story (e.g., storybooks or picture books) or informational texts 
that tell students about the world around them and answer questions. As young 
students develop their literacy abilities and are increasingly required to read 
in order to learn across the curriculum, reading to acquire information from 
books and other print materials becomes more important (Duke, 2004; Duke 
& Carlisle, 2011; Palincsar & Duke, 2004; Wharton-McDonald & Swiger, 2009). 

Aligned with these reading purposes, both the PIRLS and prePIRLS 
assessments focus on reading for interest or pleasure and reading to learn — that 
is, reading for literary experience and reading to acquire and use information. 
Because both purposes for reading are important for young students, the PIRLS 
and prePIRLS assessments contain an equal proportion of material assessing 
each purpose. However, because much online reading is done for the purpose 
of acquiring information, ePIRLS specifically focuses on reading to acquire and 
use information. 

The PIRLS and prePIRLS assessment passages are classified by their 
primary purposes, and the accompanying questions address these purposes 
for reading. That is, passages classified as literary have questions addressing 
theme, plot events, characters, and setting, and those classified as informational 
are accompanied by questions about the information contained in the passages. 
Although the assessments distinguish between purposes for reading, the 
comprehension processes readers use are more similar than different for both 
purposes; therefore, the comprehension processes are evaluated across all 
passages, including the ePIRLS online texts. 

Each purpose for reading often is associated with certain types of texts. For 
example, reading for literary experience often is accomplished through reading 
fiction, while reading to acquire and use information generally is associated with 
informative articles and instructional texts. However, the purposes for reading 
do not align strictly with text types. For example, biographies or autobiographies 
can be primarily informational or literary, but include characteristics of 
both purposes. 
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Texts often differ in the way in which ideas are organized and presented, 
eliciting a variety of ways to construct meaning (Goldman & Rakestraw, 2000; 
Kobayashi, 2002). Text organization and format can vary to a great degree, 
ranging from sequential ordering of written material to snippets of words and 
phrases arranged with pictorial and tabular data. The content, organization, 
and style that may be typical of a particular text genre have implications for 
the reader’s approach to understanding the text (Alexander & Jetton, 2000; 
Graesser, Golding, & Long, 1996; Lorch, Lemarie, & Grant, 2011; Weaver & 
Kintsch, 1996). 

As noted, it is in the interaction between reader and text that meanings 
are constructed and purposes are achieved. In selecting texts for the PIRLS 
assessments, the aim is to present a wide range of text types within each purpose 
for reading. The goal is to create a reading experience for students participating 
in each assessment that, as much as possible, is similar to authentic reading 
experiences they may have in and outside of school. 

Reading for Literary Experience 

In literary reading, readers engage with the text to become involved in events, 
setting, actions, consequences, characters, atmosphere, feelings, and ideas, and 
to enjoy language itself. In order to understand and appreciate literature, each 
reader must bring to the text his or her own experiences, feelings, appreciation 
of language, and knowledge of literary forms. For young readers, literature 
can offer the opportunity to explore situations and feelings they have not yet 
encountered. 

Events, actions, and consequences depicted in narrative fiction allow 
readers to experience vicariously and reflect upon situations that, although 
they may be imagined, illuminate those of real life. The text may present the 
perspective of the narrator or a principal character, and a more complex text 
may even have several viewpoints. Information and ideas may be described 
directly or through dialogue and events. Short stories or novels sometimes 
narrate events chronologically, or sometimes make more complex use of time 
with flashbacks or time shifts. 

The main form of literary texts used in the PIRLS and prePIRLS 
assessments is narrative fiction. Given differences in curricula and cultures 
across the participating countries, it is difficult for PIRLS to include some forms 
of literary texts. For example, poetry is difficult to translate and plays are not 
widely taught in the primary grades. 
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Reading to Acquire and Use Information 

Informational texts are both read and written for a wide variety of functions. 
While the primary function of informational text is to provide information, 
writers often address their subject matter with different objectives. Many 
informational texts are straightforward presentations of facts, such as 
biographical details or steps to accomplish a task; however, some informational 
texts are subjective. For example, authors may elect to convey facts and 
explanations through an expository summary, a persuasive essay, or a balanced 
argument. A reader must bring to these texts a critical mind in forming his or 
her own opinion. 

In order to best address the various functions of texts, information can be 
presented differently, such as by varying the content, organization, and form. 
Young students may read informational texts that cover a range of content, 
including those that are scientific, historical, geographical, or social. These 
texts also may vary in the organization of the content conveyed. For example, 
historical facts may be organized chronologically, instructions or procedures 
sequenced step-by-step, and an argument presented logically (e.g., cause and 
effect, or compare and contrast). 

Information can be presented in many different formats. Even 
informational pieces that are primarily text may include a table to document 
facts or a picture to illustrate a description. Both print materials (e.g., manuals 
and newspapers) and websites present a considerable amount of information 
via lists, charts, graphs, and diagrams. In addition, words need not be in the 
form of continuous text, such as in advertisements or announcements, or in 
sidebars to the text that offer supplemental information such as definitions, lists, 
or timelines. As noted, different presentations of textual content can demand 
that readers apply different comprehension processes. Finally, it also should be 
emphasized that a piece of informational text often incorporates one or more 
methods of presenting information. 

The informational texts used in the PIRLS assessments reflect students’ 
authentic experiences with reading informational text in and out of school. 
Typically, these passages, as well as some of the ePIRLS websites, have been 
written by authors who understand writing for a young audience, and are 
provided by the participating countries as representative of the informational 
materials their students read. 
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Processes of Comprehension 

Readers construct meaning in different ways. Therefore, PIRLS assesses four 
broad-based processes of comprehension typically used by fourth grade readers: 
focus on and retrieve explicitly stated information, make straightforward 
inferences, interpret and integrate ideas and information, and evaluate and 
critique content and textual elements. Transcending these processes are the 
metacognitive processes and strategies that allow readers to examine their 
understanding and adjust their approach (Baker & Beall, 2009; Kintsch & 
Kintsch, 2005; Paris, Wasik, & Turner, 1996; Perfetti, Landi, & Oakhill, 2005; 
Pressley, 2002; vanDijk & Kintsch, 1983). In addition, the knowledge and 
background experiences that readers bring to reading equip them with an 
understanding of language, texts, and the world, through which they filter their 
comprehension of the material (Alexander & Jetton, 2000; Beach & Hynds, 1996; 
Galda & Beach, 2001, Kintsch, 2012; 2013; Wolfe & Goldman, 2005). 

In the PIRLS assessments, the four comprehension processes are used as 
a foundation for developing the comprehension questions which are based on 
each reading passage (or set of passages). Across each assessment, the variety of 
questions measuring the range of comprehension processes enables students to 
demonstrate a range of abilities and skills in constructing meaning from written 
texts. Along with each process and its components, examples of questions that 
may be used to assess that process are discussed. 

In thinking about assessment questions, there is, of course, a substantial 
interaction between the length and complexity of the text and the sophistication 
of the comprehension processes required. Initially, it may seem that locating and 
extracting explicitly stated information would be less difficult than, for example, 
making interpretations across an entire text and integrating those with external 
ideas and experiences. However, all texts are not equal and can vary with 
regard to length, syntactic complexity, abstractness of ideas, and organizational 
structure. Thus, the nature of the text can impact the difficulty of the question 
asked, across and within the four types of comprehension processes. 

Focus on and Retrieve Explicitly Stated Information 
Readers vary the attention they give to explicitly stated information in the 
text (Flavell & Wellman, 1977; Schneider & Pressley, 1997). Some ideas in 
the text may elicit particular focus and others may not. For example, readers 
may focus on ideas that confirm or contradict predictions they have made 
about the text’s meaning or that relate to their general purpose for reading. In 
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addition, readers often need to retrieve information explicitly stated in the text 
to answer a question they bring to the reading task, or to check their developing 
understanding of some aspect of the text’s meaning. 

In focusing on and retrieving explicitly stated information, readers use 
various ways to locate and understand content that is relevant to the question 
posed. Typically, this type of text processing requires the reader to focus on 
the text at the word, phrase, and sentence level in order to construct meaning 
(Perfetti, 2007; Perfetti & Adolf, 2013). The process also may require the reader 
to focus on and retrieve pieces of information from several locations. 

Successful retrieval requires a fairly immediate or automatic understanding 
of the text (West & Stanovich, 2000). This process needs little or no inferring or 
interpreting — the meaning is evident and stated in the text. The reader must, 
however, recognize the relevance of the information or idea in relation to the 
information sought. 

Reading tasks that may exemplify this type of text processing include the 
following: 

• Identifying information that is relevant to the specific goal of reading; 

• Looking for specific ideas; 

• Searching for definitions of words or phrases; 

• Identifying the setting of a story (e.g., time and place); and 

• Finding the topic sentence or main idea (when explicitly stated). 

Make Straightforward Inferences 

As readers construct meaning from text, they make inferences about ideas or 
information not explicitly stated (Zwaan & Singer, 2003). Making inferences 
allows readers to move beyond the surface of texts and to resolve the gaps in 
meaning that often occur in texts. Some of these inferences are straightforward 
in that they are based primarily on information that is contained in the text — 
readers may merely need to connect two or more ideas or pieces of information. 
The ideas themselves may be explicitly stated, but the connection between them 
is not, and thus must be inferred. Furthermore, despite the inference not being 
explicitly stated in the text, the meaning of the text remains relatively clear. 

Skilled readers often make these kinds of inferences automatically (West 
& Stanovich, 2000). They may immediately connect two or more pieces of 
information, recognizing a relationship even though it is not stated in the text. 
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In many cases, the author has constructed a text to lead readers to an obvious 
or straightforward inference. For example, the actions of a character across the 
story may clearly point to a particular character trait, and most readers would 
arrive at the same conclusion about that character s personality or viewpoint. 

With this type of processing, readers typically focus on more than just 
word-, phrase-, or sentence-level meaning. While the focus may be on local 
meaning residing within one part of the text, the focus also may be on a more 
global meaning, representing the whole text. In addition, some straightforward 
inferences may require readers to connect local and global meanings. 

Reading tasks that may exemplify this type of text processing include the 
following: 

• Inferring that one event caused another event; 

• Concluding what is the main point made by a series of arguments; 

• Identifying generalizations made in the text; and 

• Describing the relationship between two characters. 

Interpret and Integrate Ideas and Information 
As with the more straightforward inferences, readers who are engaged in 
interpreting and integrating ideas and information in text may focus on local 
or global meanings, or may relate details to overall themes and ideas. In any 
case, these readers are making sense of the author s intent and developing a more 
complete understanding of the entire text. 

As readers interpret and integrate, they are attempting to construct a more 
specific or more complete understanding of the text by integrating personal 
knowledge and experience with meaning that resides within the text. For 
example, readers may draw on experience to infer a character’s underlying 
motive or to construct a mental image of the information conveyed. They often 
need to draw on their understanding of the world, as well as their background 
knowledge and experiences, more than they do for straightforward inferences. 

As readers engage in this interpretive process, they are making connections 
that are not only implicit, but that may be open to some interpretation based 
on their own perspective. Because of this, meaning that is constructed through 
interpreting and integrating ideas and information is likely to vary among 
readers, depending upon the experiences and knowledge they bring to the 
reading task. 
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Reading tasks that may exemplify this type of text processing include the 
following: 

• Discerning the overall message or theme of a text; 

• Considering an alternative to actions of characters; 

• Comparing and contrasting text information; 

• Inferring a story’s mood or tone; and 

• Interpreting a real-world application of text information. 

Evaluate and Critique Content and Textual Elements 

As readers evaluate the content and elements of a text, the focus shifts from 
constructing meaning to critically considering the text itself. Readers engaged 
in this process step back from a text in order to examine and critique it. 

The text content, or meaning, may be evaluated and critiqued from a 
personal perspective or with an objective view. This process may require 
readers to make a justified judgment, drawing on their interpretations and 
weighing their understanding of the text against their understanding of the 
world — rejecting, accepting, or remaining neutral to the text’s representation. 
For example, readers may counter or confirm claims made in the text or make 
comparisons with ideas and information found in other sources. 

In evaluating and critiquing elements of text structure and language, 
readers draw upon their knowledge of language usage, presentational features, 
and general or genre-specific features of texts. The text is considered as a way 
to convey ideas, feelings, and information. 

Readers may reflect on the author’s language choices and devices for 
conveying meaning and judge their adequacy. Relying on their understanding 
of language conventions, readers may find weaknesses in how the text was 
written or recognize the successful use of the author’s craft. Further, readers 
may evaluate the mode used to impart information — both visual and textual 
features — and explain their functions (e.g., text boxes, pictures, or tables). In 
evaluating the organization of a text, readers draw upon their knowledge of text 
genre and structure. The extent of past reading experience and familiarity with 
the language are essential to each piece of this process. 

Reading tasks that may exemplify this type of text processing include the 
following: 
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• Judging the completeness or clarity of information in the text; 

• Evaluating the likelihood that the events described could really happen; 

• Evaluating how likely an author’s argument would be to change what 
people think and do; 

• Judging how well the title of the text reflects the main theme; 

• Describing the effect of language features, such as metaphors or tone; 
and 

• Determining an authors perspective on the central topic. 

Introducing ePIRLS— An Assessment of Online 
Informational Reading 

A new extension of PIRLS offered for the first time in 2016, ePIRLS is an 
innovative assessment of online reading that was developed in response to the 
explosion of information availability on the Internet. As previously described, 
ePIRLS focuses on the informational reading purpose and is designed to assess 
fourth grade students’ ability to use the Internet in a school context. 

Particularly relevant to the PIRLS assessment, Internet reading is 
increasingly becoming a key component of school curricula and one of the 
central ways students are acquiring information (Leu, Kinzer, Coiro, Castek, & 
Henry, 2013; Leu, O’Byrne, Zawilinski, McVerry, & Everett-Cacopardo, 2009; 
Murnane, Sawhill, & Snow, 2012; Pew Research Center, 2012; 2013a; 2013b; 
Roswell, Kress, Pahl, & Street, 2013; Tondeur, van Braak, & Valcke, 2007). New 
digital literacies are necessary to be a successful reader on the Internet, where 
a successful reader is one that can meet his or her reading goals by efficiently 
finding and comprehending the target information (Afflerbach & Cho, 2009; 
Bawden, 2008; Coiro & Kennedy, 2011; Leu, Kinzer, Coiro, Castek, & Henry, 
2013; Leu, Kulikowich, Sedansk, & Coiro, 2008). 

Essentially, reading for informational purposes on the Internet requires 
all of the reading comprehension skills and strategies assessed by PIRLS, but 
in a different environment containing much more information. Because of the 
complexity of the Internet, online reading involves being able to use reading 
comprehension skills and strategies in contexts that are very different from those 
encountered in reading traditional printed materials as regularly assessed by 
PIRLS (Britt & Rouet, 2012; Leu, Kinzer, Coiro, Castek, & Henry, 2013). 
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ePIRLS focuses on the reading skills and strategies needed to derive 
meaning from the variety of differing presentations of online text. For example, 
Internet web pages appear different than typical printed pages. Although much 
of the Internet is devoted to providing information of one type or another, 
online presentations often use text sparingly. Similar to printed texts, web pages 
can present information in various forms, such as photos, illustrations, graphs, 
charts, tables, maps, and timelines. However, web pages also tend to be multi- 
modal in the ways they present information and contain interactive, experiential 
features that are not possible to reproduce in a print format. For example, online 
text presentations typically integrate the following dynamic elements for visual 
interest or illustration: videos and audio clips; animated graphics; pop-up 
windows with information that only appears by clicking, “hovering” above, or 
“rolling over” it; and a variety of html code-based features, such as information 
that appears and disappears, revolves, or changes color. 

The Internet also is a network of texts that are distributed across 
multiple websites and pages in a non-linear fashion. Looking for and learning 
information from the Internet involves comprehension of information arranged 
within this complex reading environment. While traditional printed text 
usually is read in a linear fashion, online reading consists of searching through 
a network of multiple texts where readers are responsible for creating their own 
paths. Readers first must access the appropriate website, and then use navigation 
strategies (e.g., multiple navigation and sub -navigation menus, tabs, and links) 
to move efficiently from within and across one web page or site to the next. 

A fundamental component of successful Internet research and 
comprehension, therefore, is the ability to locate information that meets one’s 
needs. Readers need to be able to find and select the websites that will provide 
the target information, and to navigate to the relevant web pages and also follow 
links to new websites. This may involve self-regulatory processes to maintain 
focus on the task at hand, so as not to be distracted by other interesting topics 
or advertising. 

Further, Internet searches for information require the additional 
comprehension demands of inferring the potential usefulness of yet unseen 
texts (e.g., when evaluating search engine results or links). In order to begin 
the search for information, online readers must choose among websites to find 
the one most likely to contain the target information. Once on a given website 
or page, readers must continue to infer the relevance of the various types of 
information and texts presented, while ignoring a barrage of advertising. 
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ePIRLS— Assessing the PIRLS Comprehension 
Processes in the Context of Online Informational 
Reading 

ePIRLS recognizes that online reading comprehension tasks require a 
blending of new digital literacies with traditional offline (i.e., print) reading 
comprehension processes as currently defined and assessed by PIRLS. Overall, 
the reading comprehension skills and strategies assessed in ePIRLS will parallel 
those assessed in PIRLS, with the distinction that the ePIRLS reading tasks are 
situated in a simulated Internet environment. 

The goal of ePIRLS is to assess students’ reading achievement when 
the notion of the PIRLS passages is greatly expanded to include a series of 
interconnected web pages with many different kinds of visual information, such 
as videos as well as photos, graphs, charts, and maps with dynamic graphic 
features. The websites look very different from the typical PIRLS passages, and 
involve navigating between pages and sites. 

The approach is based on using websites from the actual Internet as the 
basis for creating a closed Internet environment, through which fourth grade 
students can accomplish an online study of a science or social studies topic, 
similar to the types of projects or reports they might be asked to complete for 
school. Each task involves students working across approximately three different 
websites totaling about five to ten web pages, each containing a variety of textual 
presentations and visual displays, and including a variety of approaches to 
web navigation. 

In its simulated environment, ePIRLS incorporates a set of navigation skills 
and strategies specifically required to locate and use information on the Internet. 
These include the following: 

• Selecting websites that meet a particular information need; and 

• Using online features to locate information within websites (e.g., content 

tabs, navigation bars, graphic icons, links, and scroll bars). 

However, while ePIRLS is designed to simulate an authentic online reading 
experience, it is within a computer-based environment suitable to fourth grade 
reading levels and a timed assessment. While it is intended to reflect the types 
of online reading that students are asked to do as part of school-based projects, 
reports, and research assignments, the online environment of the ePIRLS 
assessment is necessarily very limited in comparison to the entire world of 
the Internet. 


26 


CHAPTER 1 




While recognizing that being able to locate Internet information underlies 
all of the reading processes, the emphasis in ePIRLS is on assessing reading 
comprehension rather than navigation skills. 

In order to ensure that students do not “get lost” and fail to locate the 
information that needs to be read, ePIRLS begins with a short tutorial. The 
tutorial is designed to be an equalizer across students with a range of Internet 
experiences. It covers instruction in basic point and click strategies, as well as 
how to use tabs and hyperlinks. Students also are introduced to their teacher 
avatars, Mr. and Ms. Webster. The teacher avatars guide students through the 
web pages so that students can accomplish the specific school assignments or 
research tasks in the allotted assessment time. The teacher avatars point students 
toward particular websites and provide additional assistance when students have 
difficulty locating particular web pages. 

The next sections describe how the PIRLS reading comprehension 
processes may be assessed online. It should be kept in mind that the PIRLS 
comprehension processes may be more difficult when applied to the more 
complex Internet environment. 

Focus on and Retrieve Explicitly Stated Information 
In reading a printed, linear text to retrieve specific information, the text is 
likely to be initially read and processed at a micro-level, focusing on individual 
phrases or sentences. In contrast, using online sources and search strategies can 
involve initial macro-processing. Readers need strategies for identifying the 
portion of the web page that contains the important information before they can 
focus on the sentence, phrase, or part of the graphic that has the information. 

Online reading tasks that may exemplify this type of text processing 
include the following: 

• Identifying the part of the web page that contains the information; 

• Identifying the explicitly stated information related to a specific reading 
goal; and 

• Identifying specific information on a graphic (e.g., graph, table, or map). 

Make Straightforward Inferences 

As explained previously, as readers construct meaning from text, they make 
inferences about ideas or information not explicitly stated. Online reading 
requires a considerable amount of inferencing, beginning with identifying those 
websites most likely to have the information of interest. Next, readers need 





TIMSS Er PIRLS 

International Study Center 

Lynch School of Education, Boston College 


PIRLS 2016 READING FRAMEWORK 27 


FRAMEWORK: |h 1 o' w < o - z: u 









PIRLS 



2016 



to process the information on a web page, making connections and inferring 
ideas or information not explicitly stated. Readers also may infer whether it is 
necessary or useful to follow a link to another page. 

Online reading tasks that may exemplify this type of text processing 
include the following: 

• Choosing among possible websites to identify the most appropriate, 
applicable, or useful one; 

• Filtering the content of a web page for relevance to the topic; 

• Summarizing the main intent of a web page; 

• Describing the relationship between text and graphic(s); and 

• Inferring the potential usefulness of links. 

Interpret and Integrate Ideas and Information 
Using the Internet requires the ability to read and digest information from 
multiple online sources. Integrating and synthesizing information across texts 
is very challenging, even offline, because readers need to comprehend not only 
one text, but consolidate information across two or more texts. In the Internet 
environment, this includes information presented via animation and videos as 
well as in pop-up windows and rollover text and graphics. 

Online reading tasks that may exemplify this type of text processing 
include the following: 

• Comparing and contrasting information presented within and across 
websites; 

• Relating the information in one web page or site to information in 
another web page or site; 

• Generalizing from information presented within and across web pages 
or sites; 

• Relating details from different web pages to an overall theme; and 

• Drawing conclusions from information presented in multiple websites. 

Evaluate and Critique Content and Textual Elements 
The skills required to evaluate and critique online texts in ePIRLS are very 
similar to those required for printed passages in PIRLS. However, because 
anyone can publish anything on the Internet, readers also must make judgments 
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about the credibility of the source of the information as well as determine the 
perspective, point of view, and bias in the text. In addition, the visual and textual 
features on the Internet tend to be much more varied. 

Online reading tasks that may exemplify this type of text processing 
include the following: 

• Critiquing the ease of finding information on a website; 

• Evaluating how likely the information would be to change what people 
think; 

• Describing the effect of the graphic elements on the website; 

• Determining the point of view or bias of the website; and 

• Judging the credibility of the information on the website. 

Selecting PIRLS and prePIRLS Passages and ePIRLS 
Online Texts 

The PIRLS and prePIRLS reading passages, as well as the ePIRLS online reading 
texts, undergo extensive review by the Reading Development Group and the 
National Research Coordinators. Considerable effort is expended to ensure that 
the texts have the following characteristics: 

• Clarity and coherence; 

• Appropriate content across countries and cultures; 

• Interesting, engaging content for a wide range of students; and 

• Adequate basis for assessing the full range of comprehension processes. 

In order to reflect the goal of approximating an authentic reading 
experience in the assessment, the reading passages and online materials 
presented to students must be typical of those read by students in their 
everyday experiences and reflect students’ authentic reading experiences, in 
and outside of school. In order to help achieve this goal, the texts are provided 
by the participating countries as representative of the literary and informational 
materials their students read. Texts that exist for students to read in and outside 
of school are more likely to reflect students’ ongoing reading activities and 
challenges than those written specifically for a test. 

The time constraints of the test situation place some limits on the length 
of texts, because students need time to read the entire passage and answer 
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comprehension questions. Consistent with the difference in difficulty between 
PIRLS and prePIRLS, the passages for PIRLS generally average about 800 words 
and those for prePIRLS about 400 words. However, length will vary somewhat 
because other text characteristics also affect rate of reading. 

As an additional feature to help students locate information within the 
text, items in the prePIRLS booklets are interspersed throughout each passage. 
When possible, items that require students to focus on a particular page of text 
are placed on the facing page, so that students can view both the items and the 
relevant text simultaneously. This distribution of items also helps to ensure that 
students can provide answers to some questions, even if they do not complete 
the entire passage. 

The ePIRLS online informational reading tasks in science or social studies 
are adapted from Internet websites. As described previously, each task involves 
approximately three different websites totaling about five to ten web pages. 
Reflecting the fact that online reading often involves sorting through more 
information than is actually necessary to achieve one’s goal, the texts contained 
in an ePIRLS assessment task average about 1000 words in total. 

Clarity and coherence are essential criteria for PIRLS texts. Typically, the 
passages and websites have been written by successful authors who understand 
writing for a young audience, such that the texts have an appropriate level of 
linguistic features and density of information. In the context of an international 
study, attaining authenticity in assessment reading experience may be somewhat 
constrained by the need to translate the texts into numerous languages. Thus, 
care is taken to choose texts that can be translated without loss of clarity in 
meaning, or in potential for student engagement. 

In selecting texts for use in international reading assessment, it is crucial to 
pay close attention to the potential for cultural bias. Texts that depend heavily 
on culture-specific knowledge are automatically excluded. Text selection thus 
involves collecting and considering texts from as many of the participating 
countries as possible. The goal is for the texts to be universally applicable across 
cultures, and for the set of texts in the assessment to range as widely as possible 
across nations and cultures, such that no country or culture is overrepresented 
in the assessment texts. The final selection of texts is based, in part, on the 
national and cultural representation of the entire set of assessment texts. 

The appropriateness and readability of texts for the PIRLS assessments 
primarily is determined through iterative reviews by educators and curriculum 
specialists from countries participating in the assessments. Taking into account 


30 


CHAPTER 1 




fairness and sensitivity to gender, racial, ethnic, and religious considerations, 
every effort is made to select texts that are topic and theme appropriate for the 
grade level and that elicit the full range of comprehension processes. 

Finally, it is extremely important for the texts to be interesting to the 
greatest number of students. As part of the field test, students routinely are 
asked how well they like each of the texts, and a high level of positive response 
is fundamental for a text to be selected for PIRLS. 
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PIRLS 201 6 Context 
Questionnaire Framework 

Martin Hooper, Ina V.S. Mullis, and Michael O. Martin 


PIRLS collects extensive information about the home and school contexts for 
learning to read, providing educational policy makers important insights into 
how educational systems can be improved to foster reading achievement. The 
PIRLS 2016 Context Questionnaire Framework establishes the foundation for 
the background information collected through the PIRLS 2016 Encyclopedia and 
the context questionnaires. Participating countries each contribute a chapter to 
the PIRLS 2016 Encyclopedia and the students’ parents, principals, and teachers 
complete questionnaires to provide important information about students’ home 
and school contexts for teaching and learning reading. 

The context questionnaires that accompany the reading assessment are 
an essential component of PIRLS data collection. The questionnaires cover a 
wide array of policy-relevant information about the country’s various contexts 
for teaching and learning reading. The data on home supports for learning 
as well as on educational system structure, school organization, curricula, 
teacher education, and classroom practices are considered in relation to student 
achievement and compared across countries. This information can provide 
insight into effective educational strategies for development and improvement. 

Students in the fourth year of schooling typically have gained most of 
their reading skills at school and at home. Community, school, classroom, and 
home environments that support each other can create extremely effective 
climates for learning. In order to reflect this situation, the PIRLS 2016 Context 
Questionnaire Framework encompasses five broad areas: 

• National and community contexts; 

• Home contexts; 
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• Classroom contexts; and 

• Student characteristics and attitudes toward learning. 

National and Community Contexts 

Cultural, social, political, and economic factors all contribute to the backdrop 
of a student’s literacy acquisition. At the national and community level, key 
educational policy decisions are made about how best to implement the 
curriculum, given these contextual factors. The success a country has in 
providing effective reading instruction depends on a number of interrelated 
national characteristics and decisions: 

• Language(s) and emphasis on literacy; 

• Economic resources, population demographics, and geographic 
characteristics; 

• Organization and structure of the educational system; 

• Student flow; 

• Reading curriculum in the primary grades; 

• Teachers and teacher education; and 

• Monitoring curriculum implementation. 

Language(s) and Emphasis on Literacy 

The historical background of language and literacy in a country can influence 
the challenges and instructional practices in teaching students to read. For 
example, some countries have one commonly spoken language, while others 
are historically multilingual. Immigration also can increase language diversity. 
Multilingual countries across the world have different policies for educating 
their population. Thus, decisions about the language(s) of instruction and how 
to implement those decisions can be very complicated. Studies consistently have 
shown a learning gap associated with students who do not speak the language of 
instruction in the home (Entorf & Minoiu, 2005; Schnepf, 2007; Trong, 2009). 

Economic Resources, Population Demographics, and 
Geographic Characteristics 

A country’s economic resources, demographic characteristics, and geographic 
characteristics can have a tremendous impact on the relative ease or difficulty 
of promoting literacy. 
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• Economic Resources — Countries have different levels of wealth and 
vary in how that wealth is distributed. At the national level, economic 
resources and socioeconomic equity tend to be linked to favorable 
contexts for fostering student achievement. Having economic resources 
enables better educational facilities and a greater number of well- 
trained teachers and administrators. Financial resources also provide 
the opportunity to invest in education through widespread community 
programs and by making materials and technology more readily 
available in classrooms. 

• Population Demographics — The size and diversity of a country’s 
population can increase the challenges involved in curriculum 
implementation. Some countries have a diversity of ethnic groups, 
cultures, and languages, and immigration movements can add to 
the diversity of the population. The curriculum and the educational 
system must be flexible enough to foster literacy acquisition for this 
heterogeneous population. 

• Geographic Characteristics— The sheer size of a country can pose 
challenges to curriculum implementation. This is especially true if part 
of the population is isolated in remote parts of the country. 

Organization and Structure of the Educational System 
Some countries have highly centralized educational systems in which most 
policy-related decisions are made at the national or regional level. In these 
systems, often there is a great deal of educational uniformity in terms of 
curriculum, textbooks, and general policies. Other countries have more 
decentralized systems in which many important decisions are delegated to 
local governments and schools. This decentralized structure results in greater 
variation in how schools operate and how students are taught. Research has 
found that the level of centralization of standardized assessments tends to be 
associated with greater educational equality (Van de Werfhorst & Mijs, 2010) 
and higher student outcomes (Bishop & Wofimann, 2004; Jiirges, Schneider, & 
Biichel, 2005). 

Student Flow 

Student flow refers to how students in an educational system progress through 
school. For PIRLS 2016, the student flow themes that are highly relevant include 
preprimary education, age of entry, the prevalence of grade retention, and 
student grouping. 
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• Preprimary Education — Even before they begin formal primary 
school, children may receive considerable exposure to literacy as part 
of their preprimary educational experience. As described in the PIRLS 
2011 Encyclopedia (Mullis, Martin, Minnich, Drucker, & Ragan, 2012), 
countries vary dramatically in their policies and practices with regard to 
early (preprimary) education. PIRLS 2011 supported research findings 
indicating that preprimary school can have a positive effect on academic 
achievement during primary school (Berlinski, Galiani, & Gertler, 2009; 
Tucker-Drob, 2012), with longer duration of preprimary education 
associated with higher achievement (Sammons et al., 2002). 

• Age of Entry — Policies about the age of entry to formal education (first 
year of primary school, ISCED Level 1) are important for understanding 
the variation in students’ ages across countries at the fourth grade 
(Martin, Mullis, & Foy, 2011), as are policies concerning promotion/ 
retention (see below). Typically across the PIRLS countries, students 
enter school at ages 5 to 7. Students entering school at older ages may 
have an advantage at the fourth grade for several reasons. For example, 
they have had the opportunity for more years of preprimary education 
than those students beginning school at younger ages. Also, they may 
have had the opportunity for more reading instruction upon beginning 
the first grade of primary school because they are more mature and able 
to cope with the complex cognitive demands of reading instruction. 

• Grade Retention — Grade retention practices differ among countries. 
This variation has been explained as an effect of differing educational 
policies, cultural norms, and diverging perspectives on the advantages 
of holding students back (Goos et al., 2013). Because PIRLS is a grade- 
based study, the degree of grade retention can be an important factor 
to consider when evaluating achievement results. Research has shown 
that grade retention does not have a positive relationship with student 
achievement or the emotional wellbeing of the student (Hattie, 2009; 
Jimerson, 2001). 

• Grouping for Reading Instruction — Generally, small-group instruction 
can improve reading ability (Lou et al., 2000; Puzio & Colby, 2010). 

For example, in the guided reading approach to small group reading 
instruction, teachers form small groups that are focused on instruction 
involving a specific skill or strategy rather than on reading ability alone. 
This type of flexible within-class grouping allows for differentiation in 
order to address the needs of each individual student. 
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Homogenous grouping by ability is thought to support students 
in learning at a pace that reflects their skills in the subject. However, 
research has found that grouping students by reading ability in 
elementary school is beneficial for high achieving students but has 
negative consequences for low performing students (Catsambis & 
Buttaro, 2012; Lleras & Rangel, 2009). In contrast, low ability students 
may perform best in heterogeneous groups (Lou et al., 1996). 

Reading Curriculum in the Primary Grades 

Whether formulated at the national, community, or school level, curricular 
documents define and communicate expectations for students in terms of 
the knowledge, skills, and attitudes to be developed or acquired through their 
formal reading instruction. Policies may range from those governing the grade 
in which formal reading instruction begins to those that prescribe the types of 
material and the methods to be used in teaching reading. 

Curricular aspects and governing policies particularly relevant to the 
acquisition of reading literacy include standards or benchmarks established 
for reading development, prevalence of school and classroom libraries, 
instructional time, methods and materials, and ways of identifying students in 
need of remediation. Considerable research evidence indicates that students’ 
academic achievement is closely related to the rigor of the curriculum. This 
involves a coherent progression of instruction and materials through the grade 
levels, including emphasis on decoding and comprehension strategies, and 
access to a variety of reading materials. Effective methods for disseminating 
the curriculum to teachers, parents, and the general public are important, as are 
ways for making sure that revisions and updates are integrated into instruction. 

Especially relevant to ePIRLS is the extent to which a country’s curriculum 
emphasizes online reading, digital reading, and computer skills. Some countries 
have transformed their standards and curriculum in order to address new 
media, including teaching elementary school students basic computer skills, 
such as typing and using a mouse, as well as how to retrieve information via the 
Internet, and how to use the computer for learning. Other countries place less 
emphasis on teaching these skills to young students. 

Teachers and Teacher Education 

Policies on teacher education can facilitate the successful implementation of 
the intended curriculum, and PIRLS collects information about how countries 
educate teachers in the content and pedagogical approaches specified in the 
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curriculum. As described in the PIRLS 201 1 Encyclopedia, such preparation 
and training may be an integral part of the teacher education curriculum or it 
may be included in professional development programs for practicing teachers. 
The requirements to become a primary teacher may include certain types of 
academic preparation, passing an examination, or meeting other certification 
criteria. Some countries also have induction or mentoring programs for entering 
teachers and a number of opportunities for ongoing professional development 
in order to keep teachers apprised of current developments. 

Monitoring Curriculum Implementation 

Many countries have systems in place for monitoring and evaluating curriculum 
implementation, and for assessing student achievement. Commonly used 
methods include national or regional standardized tests, school inspections, 
audits, feedback from students and parents, and teaching observations. 

Home Contexts 

Much research has provided insight into the importance of home environments 
for reading literacy development. In order to better understand the effects of the 
home and intergenerational literacy transmission, PIRLS collects data through 
both the student questionnaire and the Learning to Read Survey, completed 
by the student’s parents or caregivers. Through these two questionnaires, 
information is gathered on the following: 

• Home resources for learning; 

• Language(s) spoken in the home; 

• Parental educational expectations and academic socialization; 

• Early literacy activities and early numeracy activities; and 

• Home reading support. 

Home Resources for Learning 

Home resources for learning encompass important socioeconomic 
characteristics of the parents, such as their education level, together with home 
supports for learning and emphasis on educational activities. In educational 
research, the most influential background factors on student achievement tend 
to be those that measure socioeconomic status of the parents or caregivers, 
often indicated through proxy variables such as parental level of education, 
income, occupational class, and, more generally, home resources such as access 
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to technology, the Internet, and books, including children’s books (Bradley 
& Corwyn, 2002; Dahl & Lochner, 2005; Davis-Kean, 2005; Sirin, 2005; 
Willms, 2006). 

PIRLS has confirmed that there is a sizable association between students’ 
home resources for learning and their reading achievement. For PIRLS 2011, 
the home resources for learning scale was comprised of the following indicators: 
parents’ education, parents’ occupation, the number of books in the home, 
the number of children’s books in the home, and study supports including 
an Internet connection and the students having a room of their own (Mullis 
et al., 2012). 

Students increasingly have access to new digital media such as ebooks, 
tablets, and smart phones (Gutnick, Robb, Takeuchi, & Kotler, 2011; Rideout, 
Foehr, & Roberts, 2010). Research has shown that parents generally are 
accepting of children spending their time playing on digital media, including 
certain video games, because they believe that such activities lead to proficiency 
with computers and technology — important skills for academic and career 
success (Takeuchi, 2011). 

Research still is emerging on the relationship between access to new 
technology in the home and academic achievement in general, as well as 
increased reading literacy in particular. Research has shown general computer 
use to be associated with reading achievement (Lee, Brescia, & Kissinger, 2009). 
It is believed that, if used correctly, educational applications (apps) for mobile 
and other new media devices also can be effective, supplementary early learning 
tools for young children (Chiong & Shuler, 2010; Lieberman, Bates, & So, 2009). 
As such, there is concern that students with less home access to these costly 
resources are further disadvantaged in the classroom environment, leading to 
greater inequity in educational systems (Leu et al., 2009). 

Language(s) Spoken in the Home 

Because learning to read is dependent on children’s early language experiences, 
the language or languages spoken at home and how they are used are important 
factors in reading literacy development (Bialystok, 2006; Hoff & Elledge, 2005). 
If students are not fluent in the language of instruction, often there is at least 
an initial learning gap because students must learn the concepts and content of 
the curricula through a new language (Entorf & Minoiu, 2005; Schnepf, 2007; 
Trong, 2009), with language learners most disadvantaged in subjects with 
greater language demand, such as reading lessons (Abedi, 2002). 



TIMSS Er PIRLS 

International Study Center 

Lynch School of Education, Boston College 


PIRLS 2016 CONTEXT QUESTIONNAIRE FRAMEWORK 










PIRLS 



2016 



Parental Educational Expectations and Academic Socialization 
Parents convey their expectations to their children and provide educational 
goals for them (Hong & Ho, 2005; Jeynes, 2005). Academic socialization is 
the process of stressing the importance of education, and includes parents and 
children talking about the value of education, discussing future educational 
and occupational expectations for the child, and helping children draw links 
between schoolwork and its real-world applications (Hill & Tyson, 2009; Taylor, 
Clayton, & Rowley, 2004). 

Academic socialization also can be subject-specific. Research has found 
that parental socialization in reading is especially important in fostering student 
achievement in reading (Kloosterman, Notten, Tolsma, & Kraaykamp, 2010). 
Parents impart their own beliefs about reading that shape childrens motivation 
to read (Baker & Scher, 2002). Socialization can be subtle (e.g., young children 
seeing adults reading or using texts in different ways learn to appreciate and 
use printed material) and this process can have long term effects on a student 
academic performance (Kloosterman et al., 2010). 

Early Literacy Activities 

In addition to academic socialization, early parental involvement in childrens 
literacy activities can impact early literacy development and can have long- 
lasting effects on childrens literacy as they age (Melhuish et al., 2008; Senechal & 
LeFevre, 2002). Perhaps the most common and important early literacy activity 
involves adults and older children reading aloud with their young children 
(Federal Interagency Forum on Child and Family Statistics, 2013; Raikes et al., 
2006). By reading with children, children are asked engage with the text and 
pictures in books; they learn that printed text conveys meaning and that being 
able to read is valuable and worthwhile, and this experience can increase student 
motivation to read (Sonnenschein & Munsterman, 2002). In addition, a young 
child’s exposure to oral language is important for literacy acquisition (Hart & 
Risley, 2003). As children develop their capacity for oral language, they are 
learning the rules of language use, and this can facilitate the development of 
literacy skills. 

PIRLS has found a positive relationship between early literacy activities in 
the home and student achievement at the fourth grade. PIRLS routinely asks 
parents how often they engaged their child in early literacy activities, including 
reading books, telling stories, singing songs, playing with alphabet toys, talking 
about things they had done, talking about what they read, playing word games, 
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writing letters or words, and reading aloud signs and labels. For example, the 
PIRLS Early Literacy Activities Scale, based on these PIRLS 2011 items, was 
positively related to reading achievement in almost every country (Mullis 
et al., 2012). 

A recent study based on based on TIMSS and PIRLS 2011 data from 34 
countries, as measured by the PIRLS Early Literacy Activies Scale (Gustafsson, 
Hansen, & Rosen, 2013), also showed that engaging children in literacy 
activities was an important explanatory link in the relationship between parental 
education and later student achievement at the fourth grade. 

Home Reading Support 

After children enter formal schooling, reading activities in the home can 
complement what the child is learning in school (Darling & Westberg, 2004; Kim 
& Quinn, 2013). Parental intervention in reading has been found to be especially 
efficacious when educators train parents in specific activities that they can do 
with their child to promote literacy acquisition (Darling & Westberg, 2004; 
Senechal & Young, 2008; Van Steensel, McElvany, Kurvers, & Herppich, 2011). 
Parents also can assist their children in their literacy development by listening 
to their children when they read books aloud (Senechal & Young, 2008). 

In some countries, it is also common for parents to enroll their children 
in shadow education programs, private tutoring or classes outside of formal 
schooling that supplement the academic instruction a child receives at school 
(Bray, 1999; Stevenson & Baker, 1992). The purpose for enrolling students in this 
supplemental schooling varies. Some parents enroll students in programs for 
remedial work in order for the students to keep pace with their classmates. Other 
parents hope that additional instruction can make up for any shortcomings 
in the education provided by the child’s school (Baker, Akiba, LeTendre, & 
Wiseman, 2001). Increasingly, parents enroll students in shadow education 
programs with the goal of having their children master the curriculum in order 
to excel on an important examination (Baker & LeTendre, 2005; Buchman, 
Condron, & Roscigno, 2010), especially where students compete for limited 
educational opportunities in a tracked program (Baker & LeTendre, 2005). 

School Contexts 

A school’s environment and organization can influence the ease and effectiveness 
of reaching curricular goals. Accepting that an effective school is not simply a 
collection of discrete attributes, but rather a well-managed integrated system 
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where each action or policy directly affects all other parts, PIRLS focuses on a 
set of well-researched school quality indicators: 

• School location; 

• School composition by student socioeconomic background; 

• Instruction affected by resource shortages; 

• Teacher working conditions and job satisfaction; 

• Principal leadership; 

• School emphasis on academic success; and 

• Safe, orderly, and disciplined school. 

School Location 

Depending on the country, schools in urban areas may have access to more 
resources (e.g., museums, libraries, bookstores) than schools in rural areas. 
In some countries, schools in urban areas may provide for a more supportive 
environment because of better staffing conditions and the student population 
coming from economically more advantaged backgrounds (Erberber, 2009; 
Johansone, 2009). In contrast, in other countries, schools in urban areas are 
located in neighborhoods with considerable poverty, little community support, 
and sometimes even in areas of considerable crime and violence (Milam, Furr- 
Holden, & Leaf, 2010). 

School Composition by Student Socioeconomic Background 
Since the Coleman report (Coleman et al., 1966), there has been a great emphasis 
on how the socioeconomic status of the collective students in the school can 
influence individual student achievement (Martin, Foy, Mullis, & O’Dwyer, 
2013; Rumberger & Palardy, 2005; Sirin, 2005). The correlation between lower 
socioeconomic status and lower achievement may be able to be partially 
explained by other school factors. For example, in some countries, schools with 
students from lower socioeconomic status are taught by less qualified teachers 
(Akiba, LeTendre, & Scribner, 2007; Clotfelter, Ladd, & Vigdor, 2010). Another 
theory purports that some schools with many socioeconomically disadvantaged 
students can be overwhelmed by a culture of futility, in which education and 
schooling are viewed as an antagonistic exercise having little or no future value 
(Agirdag, Van Houtte, & Van Avermaet, 2012). 
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Instruction Affected by Resource Shortages 

The extent and quality of school resources also are critical for quality instruction 
(Greenwald, Hedges, & Laine, 1996; Lee & Barro, 2001; Lee & Zuze, 2011). 
These may include resources as basic as well-trained teachers or adequate 
classroom space and other school facilities (Schneider, 2002). Results from 
PIRLS indicate that students in schools that are well resourced generally have 
higher achievement than those in schools where shortages of resources affect 
the capacity to implement the curriculum. Two types of resources — general and 
subject-specific — affect curriculum implementation. General resources include 
teaching materials, supplies, school buildings and grounds, heating/ cooling and 
lighting systems, classroom space, audio-visual equipment such as electronic 
white boards and projectors, and computers, including tablets such as iPads. 
Subject-specific resources for reading include reading materials such as books 
and ebooks, magazines and periodicals, and digital resources such as educational 
software/applications (apps) and subscriptions to educational websites. With 
the importance of online reading for informational purposes, student access to 
computers, the Internet, and support for their online educational research are 
increasingly important to expanding literacy competencies. 

For reading, a well-resourced school library or multi-media center 
promotes student reading. The variety and richness of the reading material 
available to students forms the core of students’ reading experience in school. 
Research has shown that students use the library because there are books that 
interest them; therefore, ensuring that there are a variety of reading materials 
that would be of interest to the students at each grade is essential to promoting 
reading achievement (Clark, 2010). Libraries also are becoming multi-media 
centers, providing ebooks, access to digital periodicals, and online resources 
that allow students to seek information on subjects of interest. While school 
libraries are common in most countries, some countries have moved towards 
classroom libraries, as is discussed in the Classroom Contexts section of this 
chapter. Regardless of where the library is located, research has indicated that 
the availability of books that students can choose from is positively related to 
reading achievement (Allington et al., 2010). 

Teacher Working Conditions and Job Satisfaction 
PIRLS 2011 results showed higher achievement for schools that provide good 
working conditions for teachers. A manageable workload, adequate facilities, 
and the availability of instructional materials are important ingredients to 
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fostering productive working conditions and promoting teacher satisfaction 
(Johnson, 2006; Johnson, Kraft, & Papay, 2012). 

In addition, a positive school environment can lead to greater job 
satisfaction and teacher self-efficacy, which in turn can increase student learning 
(Caprara, Barbaranelli, Steca, & Malone, 2006). Schools can support teachers 
and increase retention by providing competitive salaries, a reasonable number 
of teaching hours, adequate workspace, and good equipment. While the physical 
conditions of the school are important, the social conditions of the school can 
be essential to retaining teachers and fostering student achievement. Important 
social factors in a school include a positive school culture, collaboration among 
teaching staff, and the leadership of the principal (Johnson et al., 2012). 

The transition from university to a school teaching position can be difficult. 
Consequently, in many countries a large percentage of new teachers leave the 
profession after only a few years of teaching (APPA, 2007; Guarino, Santibanez, 
& Daley, 2006; Hancock & Scherff, 2010). The extent to which schools take an 
active role in the acculturation and transition of new teachers may be important 
for maintaining a stable teaching force. Mentoring programs, modeling of good 
teacher practice by peers, and induction programs designed by experienced 
teachers within the school may be important aids to the beginning teacher 
(Moskowitz & Stephens, 1997; Tillmann, 2005). 

Principal Leadership 

Research has shown that a principal can affect student achievement. A 
characteristic of a successful principal is being able to articulate the mission of 
the school (Witziers, Bosker, & Kruger, 2003). As such, an effective school leader 
brings coherence to the “complexities of schooling” by aligning the structure 
and culture of a school with its core purpose (DuFour, Eaker, & DuFour, 
2005). This includes guiding the school in setting directions and seeking future 
opportunities, monitoring that the school’s goals are met, as well as building 
and sustaining an effective learning environment and a positive school climate. 
Successful principals often are involved in guiding the teaching process as 
instructional leaders and ensuring that teachers receive the necessary training 
and development to produce high achievement among the students (Robinson, 
Lloyd, & Rowe, 2008). Within the constraints of the educational system, it is the 
principal’s responsibility to ensure that the instructional time, and in particular 
the time devoted to reading, is sufficient for the purposes of curriculum 
implementation. It is also the principal’s responsibility to oversee school-level 
instructional policies, such as grouping arrangements. 
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School Emphasis on Academic Success 

Overall, the success of a school also can be attributable to a school’s emphasis on 
academic success, or the school’s expectation of academic excellence. PIRLS 2011 
results, as well as a TIMSS and PIRLS 2011 school effectiveness study (Martin, 
Foy, Mullis, & O’Dwyer, 2013), have shown an association between academic 
achievement and the school emphasis on academic success, a construct based 
on the literature on academic optimism (Hoy, Tarter, & Hoy, 2006; McGuigan & 
Hoy, 2006; Wu, Hoy, & Tarter, 2013). Indicators of school emphasis on academic 
success include school administrators’ and teachers’ expectations for successful 
curriculum implementation and student achievement, parental support for 
student achievement, and the students’ desire to achieve. 

Research also has found that teacher collaboration can increase student 
learning (Goddard, Goddard, & Tschannen-Moran, 2007; Wheelan & Kesselring, 
2005). Teachers who discuss their work with colleagues and collaborate in 
planning and implementing lessons usually feel less isolated and are less likely 
to leave teaching (Johnson, Berg, & Donaldson, 2005). The collective education 
of a school’s teachers also can be essential to its academic success. From as early 
as first grade, research has linked the collective teacher education in a school 
to student achievement (Croninger, Rice, Rathbun, &Nishio, 2007), suggesting 
that collaboration among teachers with strong educational backgrounds can 
create an emphasis on academic success within the school and facilitate the 
implementation of the curriculum. 

Collective efficacy among the teachers of the school and general trust 
that faculty members have for parents and students are additional attributes 
of a well-functioning school (Hoy et al., 2006; McGuigan & Hoy, 2006; Wu 
et al., 2013). Schools that encourage and welcome parental involvement are 
more likely to have highly involved parents than schools that do not make an 
effort to keep parents informed and participating (Jeynes, 2005). High levels 
of parental involvement can improve student achievement, as well as students’ 
overall attitude toward school (Dearing, Kreider, & Weiss, 2008; Jeynes, 2005; 
Jeynes, 2007; Taylor, Pearson, Clark, & Walpole, 2000). 

In effective schools, the principal and teachers collaborate to ensure that 
the curriculum is appropriately implemented in the classrooms. In addition to 
testing and value-added models, research has found that classroom observations 
and student surveys can provide important information about the effectiveness 
of teaching practices (Bill & Melinda Gates Foundation, 2013). 
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Safe, Orderly, and Disciplined School 

Respect for individual students and teachers, a safe and orderly environment, 
and constructive interactions among administrators, teachers, parents, and 
students all contribute to a positive school climate and lead to higher student 
achievement (Cohen, McCabe, Michelli, & Pickeral, 2009; Greenberg, Skidmore, 
& Rhodes, 2004; Konishi, Hymel, Zumbo, & Li, 2010; Martin, Foy, Mullis, & 
O’Dwyer, 2013). The sense of security that comes from having few behavioral 
problems and little or no concern about student or teacher safety at school 
promotes a stable learning environment. A general lack of discipline, especially 
if students and teachers are afraid for their safety, does not facilitate learning 
and is associated with lower academic achievement (Milam et ah, 2010; 
Stanco, 2012). Schools where there are clear rules and more fairness have 
atmospheres of greater discipline and safety (Gottfredson, Gottfredson, Payne, 
& Gottfredson, 2005). 

Bullying among students is a threat to the school learning environment. 
Bullying is aggressive behavior that is intended to harm students who are 
physically or psychologically less strong and takes a variety of forms ranging 
from name calling to inflicting physical harm. Bullying causes distress to 
victims, leads to low self-esteem, and makes victims feel like they do not 
belong (Glew, Fan, Katon, & Rivara, 2008), and research shows that bullied 
students are less likely to achieve in school (Glew et ah, 2008; Konishi et ah, 
2010; Rothon, Flead, Klineberg, & Stansfeld, 2011). With the prevalence of the 
Internet, cyberbullying is a new form of bullying that unfortunately appears to 
be common among students; and, like other bullying, cyberbullying leads to low 
self-esteem, distress, and poor achievement (Mishna, Cook, Gadalla, Daciuk, & 
Solomon, 2010; Tokunaga, 2010). Unlike bullying, the process of cyberbullying 
can be shrouded in a cloud of anonymity for the Internet bully. 

Classroom Contexts 

Because most of the teaching and learning in school takes place in the 
classroom, successful learning is influenced by the classroom environment and 
instructional activities. PIRLS 2016 focuses on the following proven practices 
that improve teaching and learning: 

• Teacher preparation and experience; 

• Classroom resources; 

• Instructional time; 
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• Instructional engagement; 

• Instruction for online reading; and 

• Classroom assessment. 

This section benefitted especially from John Hattie’s (2009) book, Visible 
Learning: A Synthesis of Over 800 Meta-analyses Relating to Achievement. 

Teacher Preparation and Experience 

The preparation and competence of teachers is critical (Darling-Hammond, 2000; 
Hill, Rowan, & Ball, 2005), and prospective teachers need coursework in order 
to gain knowledge in the subjects that they will teach, to understand about how 
students learn, and to learn about effective pedagogy in teaching reading. 

In addition to teacher education and training, teacher experience is 
essential, and the first years of teaching experience are especially important 
for teacher development (Harris & Sass, 2011; Leigh, 2010). However, research 
also has found that teachers continue to develop after five years of experience, 
and that this development can positively affect student achievement (Harris 
et al., 2011). 

Professional development through seminars, workshops, conferences, and 
professional journals can help teachers increase their effectiveness and broaden 
their knowledge (Blank & de las Alas, 2009; Yoon, Duncan, Lee, Scarloss, & 
Shapley, 2007). Professional development is especially important in order 
to train teachers in how to incorporate online reading into their classroom 
practices (Coiro, 2012). 

With education, training, and experience, teachers should feel prepared 
and confident to teach reading literacy. Research has shown that teachers’ 
confidence in their teaching skills not only is associated with their professional 
behavior, but also with students’ performance and motivation (Bandura, 1997; 
Henson, 2002). 

Classroom Resources 

Another aspect of the classroom that is relevant for reading literacy includes the 
extent of the variety and richness of the reading material available to students. 
The reading material and technology that teachers use in reading instruction 
form the core of students’ reading experience in school. 

Students who have easy access to reading materials are more likely to read. 
For this reason, many countries have moved to creating classroom libraries that 



TIMSS Er PIRLS 

International Study Center 

Lynch School of Education, Boston College 


PIRLS 2016 CONTEXT QUESTIONNAIRE FRAMEWORK 47 










PIRLS 



2016 



provide a wide variety of text and text types, including digital resources, as well 
as a special place for independent reading. It is believed that the presence of a 
classroom library can aid teachers in incorporating literature into instruction 
and foster positive reading habits and attitudes (Morrow, 2003; Routman, 2003; 
Young & Moss, 2006). 

In the digital age, a growing aspect of reading instruction is how to 
incorporate new media into reading instruction; therefore, the use of technology 
in the classroom, and teachers’ familiarity and comfort with technology, is 
increasingly important. Teachers’ decisions to use technology in the classroom 
can result from their beliefs, attitudes, and comfort levels, as well as access to 
training and materials (Mueller, Wood, Willoughby, Ross, & Specht, 2008; 
Russell, Bebell, O’Dwyer, & O’Connor, 2003). Access to technology also is an 
important factor (Hsu, Wang, & Runco, 2013). 

Instructional Time 

At the school level, the relative emphasis and amount of time specified for 
reading instruction can greatly affect the opportunities to learn. Results 
from PIRLS show that there is variation between countries in the intended 
instructional time prescribed by the curriculum and the actual time of 
implementation in the classroom. On average, however, there is very close 
agreement between the curriculum guidelines and teachers’ reports about 
implementation. Research has shown that it is especially important that 
instructional time be used effectively toward the learning goals, and not be 
spent on secondary activities unrelated to the instructional content. 

Homework is one way teachers can extend instruction and evaluate student 
learning. The types of homework assigned in reading classes regularly include 
independent reading, comprehension questions about what students have read, 
or some combination of the two. The amount of homework assigned for reading 
varies both within and across countries. In some countries, homework typically 
is assigned to students who need it the most. In other countries, students receive 
homework as an enrichment exercise. Strong students may spend less time 
on homework because they use their time more efficiently (Trautwein, 2007; 
Won & Han, 2010). For these reasons, it has been argued that the effect of 
homework may be better encapsulated by measures of homework frequency 
than homework time (Trautwein, 2007). In addition, there is evidence that 
homework is more effective for older students and higher achieving students 
(Hattie, 2009). 
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Instructional Engagement 

TIMSS and PIRLS 2011 school effectiveness research has confirmed the 
importance of student engagement with instruction as an important factor 
in predicting reading achievement (Martin, Foy, Mullis, & O’Dwyer, 2013). 
According to McLaughlin et al. (2005), student engagement focuses the 
student’s “in-the-moment” cognitive interaction with the content. “Learning 
occurs through the cognitive engagement of the learner with the appropriate 
subject matter knowledge” (McLaughlin et al., 2005, p.5). Engagement can take 
place when a student listens to the teacher, discusses texts with peers, or reads 
independently. Engagement has been conceptualized as the idea that a student’s 
“in-the-moment” mindset is torn between engagement with instruction and 
distractions that are unrelated to the topics in the class (Yair, 2000). The challenge 
for the teacher is to use effective methods of instruction in order to maintain 
student engagement in the content, activating the students cognitively (Klieme, 
Pauli, & Reusser, 2009; Lipowsky et al., 2009). A well-managed classroom and 
a supportive classroom environment can facilitate this engagement process 
(Klieme et al., 2009; Lipowsky et al., 2009). 

Effective classroom management allows for better engagement with 
teaching and learning, as well as higher achievement outcomes, because it 
focuses the class and instructional time on the topic (Fauth et al., in press; 
Lipowsky et al., 2009; Marzano, Marzano, & Pickering, 2003; Wang, Haertel, & 
Walberg, 1993). Effective teachers are strong classroom managers, who build 
trust with the students and limit disruptions to the instruction (Stronge, Ward, 
& Grant, 2011). Teachers can be strong classroom managers by ensuring that 
rules are clear, taking effective disciplinary action, building optimal student- 
teacher relationships, and maintaining an alert and objective mindset during 
instruction (Marzano et al., 2003). 

Fostering student motivation in reading is fundamental for reading 
teachers, because students who are motivated to read more, especially at a 
young age, become better readers (Lewis & Samuels, 2003). Motivation can 
be facilitated, according to self-determination theory (Deci & Ryan, 1985), by 
creating a supportive environment that fosters a sense of relatedness, competence, 
and autonomy. A classroom environment that is overly controlling can stifle 
student motivation because it removes the student’s sense of autonomy (Niemiec 
& Ryan, 2009). Effective teachers are able to create an optimal classroom 
environment by providing clear purpose and “strong guidance” for the classroom 
while encouraging cooperation among the students and an environment of 
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respect between students as well as between students and the teacher (Marzano 
et al., 2003). Supportive teacher-student relationships are important not only 
in order to foster student motivation (Cornelius-White, 2007; Marzano et 
al., 2003), but also to increase student participation and student achievement 
(Cornelius-White, 2007; Fauth et al., in press). A socially welcoming school 
environment or classroom also can provide a sense of relatedness by giving 
students a sense of belonging (Goodenow & Grady, 1993). 

Additionally, teachers can nurture student motivation in reading by 
creating an environment that allows students to work autonomously, while 
providing support and guidance (Ryan & Deci, 2000; Reeve, Jang, Carrell, Jeon, 
& Barch, 2004). Autonomy can be fostered in reading instruction by allowing 
students the opportunity to choose their reading material (Guthrie, McRae, & 
Klauda, 2007). 

An effective teacher ensures that students are actively involved in their 
own learning process. Students are more engaged in student-centered learning 
when they are working individually or with their peers rather than listening to 
a teacher lecture or watching a video (Shernoff, Csikszentmihalyi, Schneider, & 
Shernoff, 2003; Yair, 2000). Peer- tutoring, small group work, and peer mentoring 
are effective strategies that promote student engagement and are linked to 
achievement (Hattie, 2009; Springer, Stanne, & Donovan, 1999). 

Students also are more engaged when they are challenged and face 
greater cognitive demands (Shernoff et al., 2003; Yair, 2000). However, the 
challenges of the tasks should be perceived to be attainable for the students. 
In this respect, effective teaching is setting challenging yet attainable goals for 
each student and supporting the students in reaching the goals (Hattie, 2009; 
Klein, Wesson, Hollenbeck, & Alge, 1999). In setting goals, it is important that 
students understand the process of achievement, what outcome is expected, 
and why the goal is important for the learning process (Hattie, 2009; Martin, 
2006). One way that students can be supported in reaching their goals is by 
linking the new material and concepts to the students’ prior knowledge and 
understanding (Klieme et al., 2009; McLaughlin et al., 2005). Concept mapping 
(Nesbit & Adesope, 2006) and advance organizers (Hattie, 2009; Stone, 1983) 
are two proven strategies for linking prior learned concepts to new concepts. 

Discussion-based instructional approaches have been shown to be effective 
in engaging and supporting students in their reading development (Applebee, 
Langer, Nystrand, & Gamoran, 2003; Murphy, Wilkinson, Soter, Hennessey, & 
Alexander, 2009). The specific discussion-based strategy implemented by the 
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teacher (e.g. collaborative reading, questioning the author) can have a distinct 
effect on the development of a child’s reading comprehension and critical 
thinking skills, and therefore the approach should be aligned with the goals of 
the lesson (Murphy et al., 2009). 

Effective teachers also find means to emphasize the relevance of the 
learning task (Yair, 2000). By providing stimulating reading tasks around 
student interest and students’ hands-on experiences, teachers can fuel student 
interest in reading, increasing motivation and reading comprehension (Guthrie 
et al., 2006). 

Overall, research has shown that there are many strategies to teach reading 
and enhance comprehension. It is the educator’s responsiblity to understand 
the needs of the students and to incorporate the instructional techniques that 
can best foster student motivation to read and student achievement. Effective 
reading instruction provides a balanced program, integrating many components, 
including multiple texts, teacher- and student-led discussions, collaborative 
learning, time for independent reading, and a variety of assessment techniques 
(Gambrell, Malloy, & Mazzoni, 2011). Other proven instructional strategies 
include repeated reading (Therrien, 2004), phonemic awareness instruction 
(Ehri et al., 2001), and vocabulary instruction (Elleman, Lindo, Morphy, & 
Compton, 2009). 

Instruction for Online Reading 

An emerging aspect of reading instruction is the teaching of online reading. 
Unlike traditional reading, students reading online must learn to negotiate 
their way through Internet features such as hypertexts, multi-modal texts, and 
interactive texts, in order to find the information they are seeking (Coiro, 2003). 
Informational searches begin with a question, and with this question in mind 
the student uses a search engine in order to locate the information, as well as to 
identify and evaluate the credibility of the search results and the sites visited (Leu 
et al., 2007). In addition, online readers often need to synthesize the information 
from multiple websites in order to answer the question (Leu et al., 2007). While 
traditional print-based offline reading informs online reading comprehension, 
the skills needed for successful online reading go beyond those needed for print 
reading (Coiro, 2011; Leu et al., 2007). Lrom an instructor’s point of view, for 
students to become strong online readers, they not only must know how to read, 
but they also must be able to locate, evaluate, and synthesize information in the 
online environment (Coiro, 2011; Leu et al., 2007), a process that assumes basic 
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computer skills and the ability to use technology as a tool to find information 
(Tondeur, van Braak, & Valcke, 2007). Therefore, it is increasingly important for 
teachers to tailor lessons to this new form of reading (Leu et al, 2007). 

In some circumstances, poor offline readers may be more skilled or 
engaged in the interactive nature of seeking information through online media, 
and therefore can compensate for weaknesses in reading with their online skills 
(Castek, Zawilinski, McVerry, O’Byrne, & Leu, 2010; Leu et al., 2007). For 
example, students using digital technology are more likely to use resources that 
aid comprehension, such as dictionaries (Wright, Fugett, & Caputa, 2013). It 
also has been shown that students with less prior knowledge in a topic are able 
to perform well on Internet-related tasks, because their Internet skills can help 
overcome knowledge deficits (Coiro, 2011). 

Classroom Assessment 

Teachers have a number of ways to monitor student progress and achievement. 
PIRLS results show that teachers devote a fair amount of time to student 
assessment, whether as a means of gauging what students have learned or 
for providing feedback to students, teachers, and parents. The frequency and 
format of assessment are important indicators of teaching and school pedagogy, 
and research has shown that frequent testing can lead to improving student 
achievement (Ba§ol & Johanson, 2009). Informal assessments during instruction 
help teachers identify needs of particular individuals, evaluate the pace of the 
presentation, and adapt the instruction. Formal tests, both teacher-made and 
standardized assessments, typically are used to make important decisions 
about students (e.g., grades) or schools for accountability purposes. Teachers 
use a variety of formats and test a wide range of contents and cognitive skills. 
The types of questions included in tests and quizzes can send strong signals to 
students about what is important. 

Student Characteristics and Attitudes 
Toward Learning 

An important topic in educational research is the relationship between student 
attitudes toward a subject and students’ academic achievement. In educational 
research, there are numerous theories regarding how student motivation and 
confidence can lead to engagement and academic achievement. PIRLS 2016 
includes information about the following: 
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• Student readiness to learn; 

• Student motivation; 

• Student self-concept; and 

• Student reading literacy behaviors. 

Student Readiness to Learn 

In order for students to engage in a task or a goal, it is crucial that they are 
physiologically ready and possess the prerequisite knowledge to engage in the 
content (McLaughlin et al, 2005). Results from PIRLS 201 1 indicated that many 
students, even in the most developed countries, struggle to pay attention in class 
due to hunger and sleep deprivation. 

Research has identified nutritional problems to be a barrier to student 
learning, with school breakfast programs suggested as a possible solution 
(Taras, 2005). Likewise, sleep deprivation has been found to be related to 
lower achievement (Dewald, Meijer, Oort, Kerkhof, & Bogels, 2010), and may 
be associated with the early start times at certain schools (Perkinson-Gloor, 
Lemola, & Grob, 2013), as well as the socioeconomic status of the student 
(Buckhalt, 2011). 

In addition to physiological readiness, students also need to have the 
prerequisite knowledge to engage with the content because “every new thing 
that a person learns must be attached to what the person already knows” 
(McLaughlin et al., 2005, p. 5). In other words, for students to learn, they need 
to be able to connect the new content to prior knowledge. 

Student Motivation 

In addition to students’ readiness to learn, their motivation is essential to success 
in reading (Anmarkrud & Braten, 2009; Logan, Medford, & Hughes, 2011). The 
source of academic motivation and how it can be facilitated within the school, 
classroom, and home has been a recurrent area of research (Bandura, 1997; 
Csikszentmihalyi, 1990; Deci & Ryan, 1985). Students have different levels of 
motivation for each distinct task and subject area. 

Most of the literature separates motivation into two distinct constructs: 
intrinsic motivation and extrinsic motivation. Intrinsic motivation is an 
“energizer of behavior” (Deci & Ryan, 1985, p.32). Students who are intrinsically 
motivated to read find it to be interesting and enjoyable (Deci & Ryan, 1985) 
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and have a positive attitude towards reading. Although it is theorized that all 
human beings are born with intrinsic motivation to learn, contexts such as the 
home and school can either facilitate or suppress this inner motivation. 

Extrinsic motivation refers to the drive that comes from external rewards 
like praise, career success, money, and other incentives. Research consistently 
shows that intrinsic motivation is more closely related to reading achievement 
than extrinsic motivation (Becker, McElvany, & Kortenbruck, 2010; Schiefele, 
Schaffner, Moller, & Wigfield, 2012; Vansteenkiste, Timmermans, Lens, Soenens, 
& Van den Broeck, 2008). Indeed, some research points to external rewards 
dampening a student’s intrinsic motivation (Deci, Koestner, & Ryan, 1999). 
Nevertheless, most students do not have an intrinsic motivation to learn all 
subjects, and therefore fostering motivation through extrinsic rewards may be 
a necessary course of action for a teacher or a parent. In these cases, research 
has found that successful students internalize their extrinsic motivation to 
increase performance, in an environment that cultivates feelings of relatedness, 
competence, and autonomy (Ryan & Deci, 2000; Deci & Moller, 2005). 

Student Self-concept 

Students’ perceived competence in a subject is linked to their subject-specific 
self-concept. If students believe that reading tasks are outside the scope of what 
can be completed successfully, students will view the exercise as futile, and this 
will affect their motivation. In contrast, if students are confident, they are more 
likely to persevere in order to successfully complete the school task (Bandura, 
1997). Self-concept often is estimated relative to students’ peers or experiences, 
and is a multi-dimensional construct (Marsh & Craven, 2006); that is, students 
reading self-concept is distinct from their mathematics or science self-concept. 

Student Reading Literacy Behaviors 

Students who are motivated to read and have a strong reading self-concept 
tend to read more than their peers and have better reading comprehension (De 
Naeghel, Van Keer, Vansteenkiste, & Rosseel, 2012). The process can be cyclical, 
because students who are good readers, with strong reading skills, tend to read 
recreationally (Leppanen, Aunola, & Nurmi, 2005; Mol & Bus, 2011), which 
contributes to consolidating their reading ability by improving vocabularies, 
spelling abilities, and so forth (Mol & Bus, 2011). 
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Recreational reading habits often are supported by the family and friends 
of young readers; thus, a supportive home environment can be influential in 
fostering childrens reading habits. However, home support is not only important 
for children prior to their entry into primary school; home support also can be 
influential in promoting reading throughout childrens schooling (Baker, 2003; 
Klauda & Wigfield, 2012). 
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CHAPTER 3 

PIRLS 201 6 Assessment 
Design and Specifications 

Michael O. Martin, Ina V.S. Mullis, and Pierre Foy 

PIRLS 2016 consists of a wide-ranging assessment of reading comprehension to 
measure fourth grade students’ reading literacy achievement as well as a series 
of contextual questionnaires to gather information about community, home, 
and school contexts for developing reading literacy. Conducted on a regular 
five-year cycle, with each assessment linked to those that preceded it, PIRLS 
provides regular data on trends in students’ reading literacy on a common 
achievement scale. 


Student Population Assessed 

PIRLS assesses the reading literacy of children in their fourth year of formal 
schooling. This population was chosen for PIRLS because it is an important 
transition point in children’s development as readers. Typically, at this point, 
students have learned how to read and are now reading to learn. In many 
countries, this also is when students begin to have separate classes for different 
subjects, such as mathematics and science. The target population for PIRLS is 
defined as follows. 


The PIRLS target grade should be the grade that represents four 
years of schooling, counting from the first year of ISCED Level 1 . 


ISCED is the International Standard Classification of Education developed 
by the UNESCO Institute for Statistics and provides an international standard 
for describing levels of schooling across countries (UNESCO, 2012). The ISCED 
system describes the full range of schooling, from early childhood education 
(Level 0) to doctoral study (Level 8). ISCED Level 1 corresponds to primary 
education, or the first stage of basic education. The PIRLS target grade is four 
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years after the beginning of Level 1, which is the fourth grade in most countries. 
However, given the linguistic and cognitive demands of reading, PIRLS wants 
to avoid assessing very young children. Thus, if the average age of fourth grade 
students at the time of testing would be less than 9.5 years, PIRLS recommends 
that countries assess the next higher grade (i.e., fifth grade). 

Reporting Reading Achievement 

PIRLS 2016 will provide a comprehensive picture of the reading literacy 
achievement of the participating students in each country. This will include 
achievement by reading purpose and comprehension process as well as overall 
reading achievement. Consistent with the goal of a comprehensive view of reading 
comprehension, the complete PIRLS 2016 assessment consists of ten reading 
passages and accompanying questions (known as items); similarly, the prePIRLS 
assessment consists of eight less difficult reading passages and accompanying 
questions. However, in order to keep the assessment burden on any one student to 
a minimum, each student is presented with only part of the assessment according 
to a systematic booklet assembly and rotation procedure, as described in the next 
section. Following data collection, student responses are placed on a common 
reading achievement scale using item response theory methods that provide an 
overall picture of the assessment results for each country. 1 

PIRLS was designed from the outset to measure trends over time in reading 
achievement. Accordingly, the PIRLS reading achievement scale provides a 
common metric on which countries can compare their fourth grade students’ 
progress in reading over time from assessment to assessment. The PIRLS 
achievement scale was established in 2001 so that 100 points on the scale was 
equal to one standard deviation across all of the countries that participated 
in 2001, and the scale midpoint of 500 was equal to the international average 
across those countries. Using passages that were administered in both 2001 and 
2006 assessments as a basis for linking the two sets of assessment results, the 
PIRLS 2006 data also were placed on this scale so that countries could gauge 
changes in students’ reading achievement since 2001. Following a similar 
procedure, the PIRLS 2011 data also were placed on the PIRLS scale, as will 
be the data from PIRLS 2016. This will enable PIRLS 2016 countries that have 
participated in PIRLS since its inception to have comparable achievement data 
from 2001, 2006, 201 1, and 2016, and to plot changes in performance over this 
15-year period. 


1 The PIRLS scaling methodology is described in detail in Foy, Brossman, & Galia (2012). 
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In order to complement the overall reading literacy scale, and similar to 
previous PIRLS assessment cycles, PIRLS 2016 will provide separate achievement 
scales for purposes for reading and for processes of comprehension. More 
specifically, there will be two scales for reading purposes: 

• Reading for literary experience; and 

• Reading to acquire and use information. 

In addition, there will be two scales for processes of reading comprehension: 

• Retrieval and straightforward inferencing; and 

• Interpreting, integrating, and evaluating. 2 

prePIRLS 2016 also will have separate achievement scales for purposes for 
reading and for processes of comprehension. 

PIRLS 201 6 Booklet Design 

Given the broad coverage and reporting goals of the PIRLS framework and 
its emphasis on the use of a variety of authentic texts, it was inevitable that 
the specifications for the item pool would include extensive testing time. The 
PIRLS Reading Development Group found that a valid assessment of two 
purposes for reading — reading for literary experience and reading to acquire 
and use information — with reliable measures of two processes of comprehension 
required at least six hours of testing time. While the assessment material that 
can be presented in that time should provide good coverage of the reading 
material that children encounter in their everyday lives, it is not reasonable 
to expect to administer the entire set of reading passages and test items to 
any one child. Therefore, because of the difficulties of scheduling student 
assessments and because young children cannot be subjected to long testing 
periods without suffering loss of concentration and fatigue, the testing time 
is limited to 80 minutes per student, with an additional 15-30 minutes for a 
student questionnaire. 

With a total testing time for the assessment passages of more than six 
hours, but far less time available to assess any individual student, the PIRLS 
assessment materials must be divided in some way. In order to address this 
challenge, the PIRLS assessment design uses a matrix sampling technique: the 
passages and accompanying items are divided into groups or blocks, which are 
then systematically distributed among individual student booklets. 


2 Retrieval and straightforward inferencing combines items from the Focus on and Retrieve Explicitly Stated Information and 
Make Straightforward Inferences comprehension processes. Similarly, interpreting, integrating, and evaluating is based 
on items from the Interpret and Integrate Ideas and Information and Examine and Critique Content and Textual Elements 
processes. 
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In PIRLS 2016, similar to PIRLS 2006 and PIRLS 2011, the more than 
six and a half hours of testing time is divided into ten 40-minute blocks of 
passages and items, labeled L1-L5 for the literary passages and 11-15 for the 
informational texts (see Exhibit 3). Six of the ten blocks were included in 
previous PIRLS assessments: two in all three assessments (2001, 2006, and 
2011), two in both PIRLS 2006 and PIRLS 2011, and two in PIRLS 2011 only 
These “trend” blocks provide a foundation for measuring trends in reading 
achievement. Lour new blocks will be developed for use for the first time in the 
2016 assessment. 


Exhibit 3: PIRLS 2016 Matrix Sampling Blocks 


Purpose for Reading 



Block 



Literary Experience 

LI 

L2 

L3 

L4 

L5 

Acquire and Use Information 

11 

12 

13 

14 

15 


The ten blocks of passages and items in PIRLS 2016 will be distributed 
across 13 booklets following the same approach used in PIRLS 2006 and 
PIRLS 201 1 (see Exhibit 4). Each student booklet will consist of two 40-minute 
blocks of passages and items. Each student will respond to one assessment 
booklet and a student questionnaire. In order to present at least some 
passages in a more natural, authentic setting, two blocks (one literary and one 
informational) will be presented in a magazine-type format with the questions 
in a separate booklet. This booklet is referred to as the PIRLS “Reader.” 


Exhibit 4: PIRLS 2016 Student Booklet Design 


Booket 

Part 1 

Part 2 

l 

Ll 

L2 

2 

L2 

L3 

3 

L3 

L4 

4 

L4 

11 

5 

11 

12 

6 

12 

13 

7 

13 

14 

8 

14 

Ll 

9 

Ll 

11 
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Exhibit 4: PIRLS 2016 Student Booklet Design (Continued) 


Booket 

Part 1 

Part 2 

10 

12 

L2 

11 

L3 

13 

12 

14 

L4 

Reader 

L5 

15 


Booklets are distributed among students in participating classrooms so that 
the groups of students completing each booklet are approximately equivalent 
in terms of student ability. PIRLS uses item response theory scaling methods 
to assemble a comprehensive picture of the reading achievement of a country’s 
entire fourth grade student population by pooling individual students’ responses 
to the booklets that they are assigned. This approach reduces to manageable 
proportions what otherwise would be an impossible student burden, albeit at the 
cost of greater complexity in booklet assembly, data collection, and data analysis. 

In order to enable linking among booklets, at least some blocks must be 
paired with others. Because the number of booklets can become very large 
if each block is to be paired with all other blocks, it was necessary to choose 
judiciously among possible block combinations. In the PIRLS 13-booklet design, 
12 test booklets are derived by combining four literary (LI, L2, L3, and L4) and 
four informational (II, 12, 13, and 14) blocks in a variety of configurations. The 
13th booklet, the Reader, accounts for the remaining literary block, L5, and 
informational block, 15. 

In this design, each of blocks LI through L4 and II through 14 appear in 
three of the 12 booklets, each time paired with another, different, block. For 
example, as shown in Exhibit 4, literary block LI appears with literary block 
L2 in Booklet 1 and with informational blocks 14 and II in Booklets 8 and 9, 
respectively. Similarly, literary block L2 appears not only with LI in Booklet 1 
but also with literary block L3 in Booklet 2 and with informational block 12 in 
Booklet 10. 

The pairing of blocks in Booklets 1 through 12 ensures that there are good 
links both among the literary and among the informational passages and also 
between the two purposes for reading. The blocks in the Reader, L5 and 15, are 
not linked to any other blocks directly. However, because booklets are assigned 
to students using a randomized procedure, the group of students responding to 
the Reader is equivalent to those responding to the other booklets, within the 
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margin of error of the sampling process. Because each block appears in three 
of Booklets 1 through 12, the Reader is assigned three times more frequently 
in the distribution procedure than these Booklets so that the same proportion 
of students respond to blocks L5 and 15 as to each of the other literary and 
informational blocks. 

Question Types and Scoring Procedures 

Students’ ability to comprehend text through the four PIRLS comprehension 
processes is assessed via comprehension questions that accompany each text. 
Two question formats are used in the PIRLS assessment: multiple-choice and 
constructed-response. Each multiple-choice question is worth one point. 
Constructed-response questions are worth one, two, or three points, depending 
on the depth of understanding required. Up to half of the total number of points 
represented by all of the questions will come from multiple-choice questions. 
In the development of comprehension questions, the decision to use either a 
multiple-choice or a constructed-response format is based on the process being 
assessed, and on which format best enables test takers to demonstrate their 
reading comprehension. 

Multiple-choice Questions 

Multiple- choice questions provide students with four response options, of which 
only one is correct. Multiple-choice questions can be used to assess any of the 
comprehension processes. However, because they do not allow for students’ 
explanations or supporting statements, multiple-choice questions may be less 
suitable for assessing students’ ability to make more complex interpretations 
or evaluations. 

In assessing fourth grade students, it is important that linguistic features of 
the questions be developmentally appropriate. Therefore, questions are written 
clearly and concisely. Response options also are written succinctly in order to 
minimize the reading demand of the question. Incorrect options are written to 
be plausible, but not deceptive. For students who may be unfamiliar with this 
test question format, the instructions given at the beginning of the test include 
a sample multiple- choice item that illustrates how to select and mark an answer. 

Constructed-response Questions 

Constructed-response test items require students to provide a written response, 
rather than select a response from a set of options. The emphasis placed on 
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constructed-response questions in the PIRLS assessment is consistent with 
the definition of literacy underlying the framework. It reflects the interactive, 
constructive view of reading — meaning is constructed through an interaction 
between the reader, the text, and the context of the reading task. This question 
type may be used to assess any of the four comprehension processes. However, 
it is particularly well suited for assessing aspects of comprehension that require 
students to provide support or that result in interpretations depending upon 
students’ background knowledge and experiences. 

In the PIRLS assessment, constructed-response questions may be worth 
one or two points (short-answer items), or three points (extended-response 
items), depending on the depth of understanding or the extent of textual 
support the question requires. In framing these questions, it is important to 
provide enough information to help students clearly understand the nature of 
the response expected. 

Each constructed-response question has an accompanying scoring guide 
that describes the essential features of appropriate and complete responses. 
Scoring guides focus on evidence of the type of comprehension the questions 
assess. The guides describe evidence of partial understanding and evidence of 
complete or extensive understanding. In addition, sample student responses at 
each level of understanding provide important guidance to scoring staff. 

In scoring students’ responses to constructed-response questions, the focus 
is solely on students’ understanding of the text, not on their ability to write well. 
Also, scoring takes into account the possibility of various interpretations that 
may be acceptable, given appropriate textual support. Consequently, a wide 
range of answers and writing ability may appear in the responses that receive 
full credit to any one question. 

Score Points 

In developing the PIRLS and prePIRLS assessments, the aim is to create blocks 
that each provide, on average, at least 15 score points consisting of the following: 
approximately seven multiple-choice items (1 point each), two or three short- 
answer items (1 or 2 points each), and one extended-response item (3 points). 
Items in each block should address the full range of PIRLS comprehension 
processes. The exact number of score points and the exact distribution of 
question types per block will vary somewhat, because different texts yield 
different types of questions. 
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Releasing Assessment Materials to the Public 

An essential aspect of the PIRLS design for measuring trends over time in 
reading achievement is that, with each cycle, PIRLS releases a number of 
passages and items into the public domain to help readers understand as much 
as possible about the content and approach of the assessment. At the same time, 
a number of passages and items are retained and kept confidential to be used in 
future assessments as the basis for measuring trends. As passages and items are 
released, new assessment materials are developed to take their place. 

According to the PIRLS design, four blocks were released following the 
PIRLS 2011 data collection, two from the 2006 assessment (L2 and 12), and 
two from those developed for 2011 (L5 and 15). These released passages and 
items may be found in the PIRLS 201 1 International Results in Reading (Mullis, 
Martin, Foy, & Drucker, 2012). Following the publication of the international 
report for PIRLS 2016, a further four blocks will be released: two that were 
used in both the 2011 and 2016 assessments, and two from those developed for 
specifically for PIRLS 2016. 

prePIRLS 2016 Booklet Design 

The prePIRLS assessment follows the PIRLS student populations and assessment 
design as closely as possible, with changes made only when appropriate for the 
prePIRLS context. Also similar to PIRLS, prePIRLS provides a broad picture of 
students’ reading achievement by placing student responses on a common scale 
and measuring trends in achievement over time. 

Like PIRLS, prePIRLS will report student achievement on an overall reading 
literacy scale and separate achievement scales for the literary and informational 
reading purposes. The reading comprehension process scales, however, differ 
from PIRLS due to the greater emphasis on the process of focusing on and 
retrieving explicitly stated information in the prePIRLS items. More specifically, 
the two scales for the processes of comprehension will be as follows: 

• Retrieving explicitly stated information; and 

• Inferencing, interpreting, and evaluating. 3 

Following the design that has proven effective in previous cycles of PIRLS, 
prePIRLS also uses a matrix sampling technique for the administration of the 
assessment. prePIRLS consists of eight blocks of reading passages and their 
accompanying items, for a total of five and a half hours of testing time (see 


3 Retrieving explicitly stated information uses items from the Focus on and Retrieve Explicitly Stated Information 

comprehension process. Inferencing, interpreting, and evaluating is based on items from the Make Straightforward 
Inferences, Interpret and Integrate Ideas and Information, and Examine and Critique Content and Textual Elements processes. 
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Exhibit Z). Four of the passage and item blocks were previously used in 201 1 
and will be used again in 2016 as a basis for measuring trends, and four will be 
developed for use in 2016 for the first time. 

The prePIRLS 2016 booklet design is based on the PIRLS design, but has 
eight rather than ten passage/ item blocks and does not incorporate a Reader 
booklet (see Exhibit 5). Accordingly, the prePIRLS booklet design consists of 12 
booklets, based on prePIRLS literary blocks LI through L4 and informational 
blocks II through 14. The prePIRLS booklet design uses the same block pairings 
as PIRLS booklets 1-12, as shown in Exhibit 6. 


Exhibit 5: prePIRLS 2016 Matrix Sampling Blocks 


Purpose for Reading 


Block 


Literary Experience 

LI 

L2 

L3 

L4 

Acquire and Use Information 

11 

12 

13 

14 


Exhibit 6: prePIRLS 2016 Student Booklet Design 


Booket 

Part 1 

Part 2 

l 

Ll 

L2 

2 

L2 

L3 

3 

L3 

L4 

4 

L4 

11 

5 

11 

12 

6 

12 

13 

7 

13 

14 

8 

14 

Ll 

9 

Ll 

11 

10 

12 

L2 

11 

L3 

13 

12 

14 

L4 


As in PIRLS, each prePIRLS booklet contains two 40-minute test blocks 
for a total of 80 minutes of testing time per student, followed by 15-30 minutes 
for a student questionnaire. 

Similar to PIRLS, prePIRLS strives to replicate an authentic reading 
experience with materials that are engaging and familiar to the students 
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participating in prePIRLS. However, sometimes it is necessary to adapt texts 
in order to ensure that the subject matter is age-appropriate, while keeping 
the reading demand reasonable for prePIRLS students. Passages selected for 
prePIRLS generally average 400 words in length in order to ensure that students 
have ample time to read the passage and respond to the accompanying items. 
As an additional step to help students locate information within the text, items 
are interspersed throughout the passage. When possible, items that require 
students to focus on a particular page of text are placed on the facing page, so 
that students can view both the items and the relevant text simultaneously. This 
distribution of items also helps to ensure that students can provide answers to 
some questions, even if they do not complete the entire passage. 

The prePIRLS items use multiple- choice and constructed-response formats, 
as in PIRLS, though constructed-response items usually are worth only one 
or two points. However, there is a slightly higher percentage of constructed- 
response items in the prePIRLS assessment, comprising up to 60 percent of the 
total score points. This decision was made because constructed-response items 
that require a very short response often are easier for early readers due to the 
lighter reading demand, as compared with multiple-choice items that require 
students to read and evaluate four response options. In addition, multiple-choice 
items may lose some of their effectiveness in passages as short as those used in 
prePIRLS, because there are fewer plausible distracters that can be drawn from 
the text. 

ePIRLS 201 6 Design 

The ePIRLS computer-based assessment of online reading is designed as an 
extension to PIRLS that measures student informational reading in an online 
environment. ePIRLS is administered by computer, and requires students to use 
a mouse or other pointing device to navigate through the assessment and to use 
a computer keyboard to type their responses to the assessment questions. All 
students participating in ePIRLS also are expected to have participated in PIRLS. 

The complete ePIRLS assessment consists of four school-based online 
reading tasks, each of which involves 2-3 different websites totaling 5 to 10 
web pages together with a series of comprehension questions based on the task. 
Similar to the PIRLS and prePIRLS passages, each task with accompanying 
questions is expected to take 40 minutes to complete, and each individual 
student will complete two ePIRLS tasks, followed by 15-30 minutes for a short 
online questionnaire. 

With each student completing two of the four tasks, ePIRLS uses a matrix- 
sampling design in order to establish four task combinations, as shown in 
Exhibit 7. 
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Exhibit 7: ePIRLS 2016 Student Task Combinations 


Student Task Combination 

First Task 

Second Task 

Task Combination #1 

E01 

E02 

Task Combination #2 

E02 

E03 

Task Combination #3 

E03 

E04 

Task Combination #4 

E04 

E01 


Following the usual PIRLS strategy, the four ePIRLS task combinations 
are distributed among students in participating classrooms so that the groups 
of students completing each task combination are approximately equivalent 
in terms of student ability. ePIRLS uses item response theory scaling methods 
to assemble a comprehensive picture of the online informational reading 
achievement of a country’s fourth grade student population by pooling 
individual students’ responses to the tasks that they have been assigned. 

Because 2016 is the inaugural year for ePIRLS, all four tasks are newly 
developed. Two of the tasks will be kept secure after the 2016 assessment in 
order to measure trends in future assessment cycles, and two will be released 
to the public. 

Background Questionnaires 

An important purpose of PIRLS and prePIRLS is to study the community, home, 
and school factors associated with children’s reading literacy by the fourth 
grade. To that end, PIRLS and prePIRLS 2016 will administer questionnaires 
to students, their parents, their teachers, and the principals of their schools. 
The questions are designed to measure key aspects of students’ home and 
school environments. PIRLS and prePIRLS 2016 also administer curriculum 
questionnaires to specialists in order to collect information about educational 
policies and the national contexts that shape the scope and implementation of 
reading curricula across countries. 

Student Questionnaire 

A questionnaire is completed by each student who participates in the PIRLS or 
prePIRLS reading assessment. This questionnaire asks about aspects of students’ 
home and school lives, including demographic information, home environment, 
school climate for learning, out-of-school reading behaviors, classroom 
engagement, and attitudes toward reading. The student questionnaire requires 
15-30 minutes to complete. Students participating in ePIRLS also will complete 
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a short online questionnaire about their experience with online reading at home 
and in school, in addition to the regular PIRLS student questionnaire. 

Learning to Read Survey (Home Questionnaire) 

This short questionnaire is addressed to the parents or primary caregivers 
of each student taking part in the PIRLS 2016 data collection. It asks about 
language spoken in the home, preschool experiences, homework activities, 
home-school involvement, books in the home, and parents’ education and 
occupation. In addition, this questionnaire collects information on early literacy 
and numeracy activities, reading and quantitative readiness, and parents’ reading 
activities and attitudes toward reading. Together with information collected 
from the students, parents’ responses will provide a more complete picture of 
an important context for learning to read. This questionnaire is designed to take 
10-15 minutes to complete. 

Teacher Questionnaire 

The reading teacher of each fourth grade class in PIRLS 2016 will be asked to 
complete this questionnaire, which is designed to gather information about 
teacher characteristics and classroom contexts for developing reading literacy. 
The questionnaire asks teachers about their background and education, the 
school climate for learning, attitudes toward teaching, classroom characteristics, 
and strategies for student engagement. It also asks about the following: reading 
instructional time, approaches, activities, and materials; computer and library 
resources; homework; and preparation to teach reading. This questionnaire 
requires about 30 minutes of the teacher’s time. 

School Questionnaire 

The principal of each school in PIRLS 2016 will be asked to respond to this 
questionnaire. It asks about school characteristics, instructional time, resources 
and technology, parental involvement, school climate for learning, teaching staff, 
the role of the principal, and students’ reading readiness. It is designed to take 
about 30 minutes. 

Curriculum Questionnaire 

In order to provide information about the goals of reading instruction, the 
national research coordinator in each country participating in PIRLS and 
prePIRLS 2016 will complete a questionnaire about the country’s reading 
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curriculum, including national policy on reading, goals and standards for 
reading instruction, time specified for reading, and provision of books and other 
literary resources. 

PIRLS 2016 Encyclopedia 

The PIRLS 2016 Encyclopedia will provide a profile of each country’s education 
system, with a particular focus on reading education for primary school 
children. This volume will provide general data on economic and educational 
indicators and will describe how the education system is organized as well as 
how decisions about education are made. The reading curriculum, including 
goals, materials, and instruction, will be discussed, along with information on 
assessment of reading achievement. 
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APPENDIX B 

Sample PIRLS Passages, 
Questions, and Scoring Guides 


Reading for Literary Experience 

Enemy Pie 


Reading to Acquire and Use Information 

The Mystery of the Giant Tooth 
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Enemy Pie 

by Derek Munson 
illustrated by Tara Calahan King 

It was a perfect summer until Jeremy Ross moved in right next door to 
my best friend Stanley. I did not like Jeremy. He had a party and I wasn’t 
even invited. But my best friend Stanley was. 

I never had an enemy until Jeremy moved into the neighborhood. Dad 
told me that when he was my age, he had enemies, too. But he knew of a 
way to get rid of them. 

Dad pulled a worn-out 
scrap of paper from a recipe 
book. 

“Enemy Pie,” he said, 
satisfied. 

You may be wondering 
what exactly is in Enemy 
Pie. Dad said the recipe was 
so secret, he couldn’t even 
tell me. I begged him to tell 
me something — anything. 

“I will tell you this, 

Tom,” he said to me. “Enemy 
Pie is the fastest known way to get rid of enemies.” 

This got me thinking. What kinds of disgusting things would I put into 
Enemy Pie? I brought Dad earthworms and rocks, but he gave them right 
back. 
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I went outside to play. All the while, I listened to the sounds of my 
dad in the kitchen. This could be a great summer after all. 


I tried to imagine how horrible Enemy Pie must smell. But I smelled 
something really good. As far as I could tell, it was coming from our kitchen. 
I was confused. 

I went inside to ask Dad what was wrong. Enemy Pie shouldn’t smell 
this good. But Dad was smart. “If it smelled bad, your enemy would never 
eat it,” he said. I could tell he’d made this pie before. 

The oven buzzer rang. Dad put on oven mitts and pulled out the pie. It 
looked good enough to eat! I was beginning to understand. 

But still, I wasn’t sure how this Enemy Pie worked. What exactly did 
it do to enemies? Maybe it made their hair fall out, or their breath stinky. I 
asked Dad, but he was no help. 

While the pie cooled, Dad filled me in on my job. 

He whispered. “In order for it to work, you need to spend a day with 
your enemy. Even worse, you have to be nice to him. It’s not easy. But that’s 
the only way that Enemy Pie can work. Are you sure you want to do this?” 

Of course I was. 

All I had to do was spend one day with Jeremy, then he’d be out of my 
life. I rode my bike to his house and knocked on the door. 

When Jeremy opened the 
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He looked confused. “I’ll go ask my mom,” he said. He came back with 
his shoes in his hand. 

We rode bikes for awhile, then ate lunch. After lunch we went over to 
my house. 

It was strange, but I was having fun with my enemy. I couldn’t tell Dad 
that, since he had worked so hard to make the pie. 

We played games until my dad called us for dinner. 

Dad had made my favorite food. It was Jeremy’s favorite, too! Maybe 
Jeremy wasn’t so bad after all. I was beginning to think that maybe we 
should forget about Enemy Pie. 

“Dad”, I said, “It sure is nice having a new friend.” I was trying to tell 
him that Jeremy was no longer my enemy. 

But Dad only smiled and nodded. I think 
he thought I was just pretending. 

But after dinner, Dad brought out the 
pie. He dished up three plates and passed 
one to me and one to Jeremy. 

“Wow!” Jeremy said, looking at the pie. 

I panicked. I didn’t want Jeremy to eat 
Enemy Pie! He was my friend! 

“Don’t eat it!” I cried. “It’s bad!” 

Jeremy’s fork stopped before reaching 
his mouth. He looked at me funny. I felt 
relieved. I had saved his life. 
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“If it’s so bad,” Jeremy asked, “then why has your dad already eaten 
half of it?” 

Sure enough, Dad was eating Enemy Pie. 

“Good stuff,” Dad mumbled. I sat there watching them eat. Neither one 
of them was losing any hair! It seemed safe, so I took a tiny taste. It was 
delicious! 

After dessert, Jeremy invited me to come over to his house the next 
morning. 

As for Enemy Pie, I still don’t know how to make it. I still wonder if 
enemies really do hate it or if their hair falls out or their breath turns 
bad. But I don’t know if I’ll ever get an answer, because I just lost my best 
enemy. 
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Questions Enemy Pie 

1. Who is telling the story? 

© Jeremy 

® Dad 
© Stanley 
(d) Tom 

2. At the beginning of the story, why did Tom think Jeremy was his 
enemy? 



3. Write one ingredient that Tom thought would be in Enemy Pie. 
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4. Find the part of the story next to the picture of a piece of pie: 
Why did Tom think it could be a great summer after all? 

(a) He liked playing outside. 

© He was excited about Dad’s plan. 

© He made a new friend. 

© He wanted to taste Enemy Pie. 

5. How did Tom feel when he first smelled Enemy Pie? Explain why 
he felt this way. 



6. What did Tom think could happen when his enemy ate Enemy Pie? 
Write one thing. 



© 


© 


© 


© 


© 
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7. What were the two things Tom’s dad told Tom to do for Enemy Pie 
to work? 



8. Why did Tom go to Jeremy’s house? 

(a) To invite Jeremy to dinner. 

® To ask Jeremy to leave Stanley alone. 

© To invite Jeremy to play. 

© To ask Jeremy to be his friend. 

9. What surprised Tom about the day he spent with Jeremy? 
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10. At dinner, why did Tom begin to think he and his dad should 
forget about Enemy Pie? 

(a) Tom did not want to share dessert with Jeremy. 

(j[) Tom did not think Enemy Pie would work. 

© Tom was beginning to like Jeremy. 

( 3 ) Tom wanted to keep Enemy Pie a secret. 


11. How was Tom feeling when Dad passed the piece of Enemy Pie to 
Jeremy? 

© alarmed 

© satisfied 

© surprised 

© confused 
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12. What was it about Enemy Pie that Dad kept secret? 

(a) It was a normal pie. 

(j[) It tasted disgusting. 

© It was his favorite food. 

( 3 ) It was a poisonous pie. 

13. Look at this sentence from the end of the story: 

“After dessert, Jeremy invited me to come over to his house the 
next morning.” 

What does this suggest about the boys? 

® They are still enemies. 

® They do not like to play at Tom’s house. 

© They wanted to eat some more Enemy Pie. 

© They might be friends in the future. 

14. Use what you have read to explain why Tom’s dad really made 
Enemy Pie. 



108 


APPENDIX B 




15. What kind of person is Tom’s dad? Give an example of what he did 
in the story that shows this. 



16. What lesson might you learn from this story? 
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Scoring Guides for Constructed-response Questions 

Enemy Pie, Item 2 

2. At the beginning of the story, why did Tom think Jeremy was his enemy? 

Process: Make Straightforward Inferences 

1 - Acceptable Response 

The response shows understanding that Tom considered Jeremy his enemy either because Jeremy did not 
invite him to his party, or because Jeremy invited Tom’s best friend Stanley and not him. 

Examples: 

Tom was not invited to Jeremy’s party 

Jeremy invited his friend to his party, but did not invite Tom. 

Or, the response shows understanding that Tom was afraid that Jeremy would take his place as Stanley’s 
best friend. 

Examples: 

Tom was jealous of him moving in next to Stanley. 

Jeremy took his best friend. 

0 - Unacceptable Response 

The response does not show understanding of why Tom considered Jeremy his enemy. The response may 
repeat words from the question, or may provide a vague response that acknowledges that Jeremy moved in 
next door to Stanley or invited him to his party without showing understanding of the consequence. 
Examples: 

Jeremy was his enemy. 

Jeremy moved in right next door to Tom’s best friend. 

Jeremy invited Stanley to his party. 

Jeremy was new in the neighborhood. 

Jeremy was his friend. 
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Enemy Pie, Item 3 

3. Write one ingredient that Tom thought would be in Enemy Pie. 

Process: Focus on and Retrieve Explicitly Stated Information 


1 - Acceptable Response 


The response identifies either (earth)worms or rocks as an ingredient. 

NOTE TO SCORERS: Do not credit responses that include ANY incorrect piece(s) of information 
alongside correct answers. 

Answers: 

earthworms 

worms 

rock 


0 - Unacceptable Response 


The response does not provide either of the ingredients listed above. The response may provide a vague 
description without mention of a specific ingredient, may name an incorrect ingredient alongside a correct 
response, or may describe what would happen to someone who ate the pie. 

Examples: 
rocks and dirt 


worms and raspberries 
disgusting things 
secret ingredients 


things that make your hair fall out 
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Enemy Pie, Item 5 

5. How did Tom feel when he first smelled Enemy Pie? Explain why he felt this way. 

Process: Make Straightforward Inferences 

2 - Complete Comprehension 

The response shows understanding that Tom was confused because he thought Enemy Pie was supposed to 
smell bad, or that Tom was surprised because the pie his dad made (actually) smelled good. 

NOTE TO SCORERS: Students may express Tom’s confused or surprised feelings in a variety of 
ways. 

Examples: 

confused because he thought it was made with disgusting things 
He didn’t understand. It should taste horrible. 

He felt unsure. Enemy Pie should smell bad. 
surprised because it smelled really good 

1 - Partial Comprehension 

The response shows understanding that Tom was confused or surprised when he smelled Enemy Pie for the 
first time, but does not explain why. 

Examples: 

confused 

He wondered what was going on. 

Or, the response explains that Enemy Pie didn’t smell the way he thought it would without providing the 
feeling. 

Examples: 

Enemy Pie shouldn’t smell this good. 

He thought the pie would smell bad. 

He thought it would smell awful, but it didn’t. 

0 - No Comprehension 

The response does not provide either the appropriate feeling or an explanation. 

Examples: 

He smelled something really good. (Please note that this response does not provide a feeling or a clear 
explanation for why Tom was confused.) 

He felt hungry. 
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Enemy Pie, Item 6 


6. What did Tom think could happen when his enemy ate Enemy Pie? Write one 
thing. 

Process: Focus on and Retrieve Explicitly Stated Information 


1 - Acceptable Response 


The response identifies one of the consequences of eating Enemy Pie from the list below. 

NOTE TO SCORERS: Ignore minor variations in phrasing from the text, as long as it is clear 
what is intended. 

Consequences of Eating Enemy Pie: 

His hair would fall out. 

His breath would stink. 

He would go away. 

Something bad would happen./He would get sick (or die). 

0 - Unacceptable Response 


The response does not provide any of the words or phrases in the list above. The response may repeat words 
from the question. 

Examples: 

He might like it. 

He would become his friend. 


Nothing would happen. 

He would become his enemy. 
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Enemy Pie, Item 7 

7. What were the two things Tom’s dad told Tom to do for Enemy Pie to work? 

Process: Focus on and Retrieve Explicitly Stated Information 

2 - Complete Comprehension 

The response identifies both actions that make Enemy Pie work: 1) spending the day with his enemy and 2) 
being nice to him. 

NOTE TO SCORERS: Any responses that do not include specific reference to the amount of time 
that should be spent (a day ) should not be credited. 

Examples: 

be nice to his enemy for a whole day 
spend the whole day with Jeremy and be nice 
be nice and play with him for a day 
play all day with Jeremy and be friendly 

1 - Partial Comprehension 

The response provides one action that Tom was told to do by his Dad. 

Examples: 
be nice 

spend the day with him 
play and be nice 

0 - No Comprehension 

The response does not provide an accurate action that Tom was told to do by his Dad. 

Examples: 

play with him (Please note that this is not one of the things Tom’s dad told him to do and is too vague to be 
considered as a paraphrase of either spending the day or being nice.) 

stop being enemies (Please note that Tom’s dad did not tell him to stop being enemies with Jeremy, nor did 
he tell him to be his friend.) 

invite him over for dinner 
eat Enemy Pie 
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Enemy Pie, Item 9 

9. What surprised Tom about the day he spent with Jeremy? 

Process: Make Straightforward Inferences 

1 - Acceptable Response 

The response shows understanding that Tom had a positive experience with Jeremy. The response may 
indicate that he enjoyed spending time with Jeremy, that Jeremy wasn’t as bad as Tom expected, or that they 
had become friends. 

Examples: 

He was actually having fun with Jeremy. 

They were getting along. 

Jeremy wasn’t so bad after all. 

Jeremy was nice. 

They became friends. 

It was a good day. 

0 - Unacceptable Response 

The response does not accurately describe what surprised Tom. 

Examples: 

Tom was surprised. 

Jeremy was going to eat the Enemy Pie. 
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Enemy Pie, Item 14 

1 4. Use what you have read to explain why Tom’s dad really made Enemy Pie. 

Process: Interpret and Integrate Ideas and Information 

1- Acceptable Response 

The response demonstrates understanding that Tom’s dad’s plan for Enemy Pie was for Tom and Jeremy to 
become friends. 

NOTE TO SCORERS: The response does not need to explicitly state that Tom's dad made them 
spend time together to he awarded credit. 

Examples: 

to make them be friends and not enemies 
He wanted them to be friends. 

to get them to play together and to make them friends 

He wanted them to be friends so he got them to play with each other. 

to play a trick for Tom to see that Jeremy was nice after all (Please note that this is an acceptable 
paraphrase of the boys becoming friends.) 

0 - Unacceptable Response 

The response does not provide an appropriate explanation for why Tom’s dad really made Enemy Pie. The 
response may indicate that Tom’s dad wanted the boys to spend time together without specific reference to 
the intended outcome, or it may refer generally to Tom having no enemies without reference to Tom and 
Jeremy’s relationship. 

Examples: 

He made Tom play with Jeremy. 

So they would get to know each other. 

He thought it would work and make Jeremy leave. 

He made the pie for them all to share. 
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Enemy Pie, Item 15 


1 5. What kind of person is Tom’s dad? Give an example of what he did in the story 
that shows this. 

Process: Interpret and Integrate Ideas and Information 


2 - Complete Comprehension 


The response describes one plausible character trait of Tom’s dad that is central to his role in the story (e.g., 
helpful, caring, nice, good, smart, clever, tricky, secretive). In addition, the response provides one example 
of Tom’s dad’s actions that is evidence of the character trait. 

NOTE TO SCORERS: Traits may be expressed as a longer description, rather than as a single 
word. 

Examples: 

He was caring because he wanted to help his son make friends. 

He was smart in how he found a way for the boys to like each other. 

He was the kind of person who kept secrets. He kept Tom from finding out that Enemy Pie was just a 
normal pie. 

He was nice. He wanted Tom and Jeremy to get along. 

Tom’s dad was kind. He thought of a plan for his son to make friends. 

1 - Partial Comprehension 


The response provides one plausible character trait of Tom’s dad that is central to his role in the story (e.g., 
helpful, caring, smart, clever, tricky, secretive). Traits maybe expressed as a longer description, rather than 
as a single word. 

Examples: 

He was caring. 

He was nice. 


He was a good person. 
He was a good dad. 

He cared about his son. 


He wanted to help Tom. 

He was clever. He made a pie. (Please note that ‘he made a pie is not an appropriate example of Tom’s dad’s 
cleverness.) 


0 - No Comprehension 


The response does not provide an appropriate description of Tom’s dad’s character. The response may 
provide a general character trait of Tom’s dad that is not supported by the text, or a vague description that 
demonstrates limited comprehension of the story without further textual support. 
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Examples: 

Tom’s dad was mean. 

He was confused. (Please note that this response describes Tom in the story.) 

He was a cook. He baked a pie. (Please note that ‘he was a cook’ is not a character description.) 

Or, the response may provide an example of Toms dad’s actions without providing a character trait. 
Examples: 

He made Tom think Enemy Pie would work. 

He kept the recipe a secret. 

He told Tom to play with Jeremy. 


Enemy Pie, Item 16 

1 6. What lesson might you learn from this story? 

Process: Examine and Evaluate Content, Language, and Textual Elements 

1- Acceptable Response 

The response provides an evaluation of the main message or theme of the story that acknowledges the 
importance of giving a relationship the chance to grow before deciding whether someone is your friend, or 
indicates that it is possible to change how you feel about someone. 

Examples: 

Don’t judge someone before you know them. 

You can make friends if you give it a chance. 

Your enemy can become your friend. 

Try to like your enemy. They might become your friend. 

0 - Unacceptable Response 

The response does not provide a plausible evaluation of the main message or theme of the story. The 
response may provide a main message that is too general, or may refer to a message that is not central to the 
story. 

Examples: 

Be nice to everyone. 

You shouldn’t have enemies. (Please note that this is an inaccurate generalization of the main message.) 
Don’t eat Enemy Pie. 

It isn’t nice to exclude someone from your party. 
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The GIANT Tooth 

]VIystery 

A fossil is the remains of any creature or plant that lived on the 
Earth many, many years ago. People have been finding fossils for 
thousands of years in rocks and cliffs and beside lakes. We now 
know that some of these fossils were from dinosaurs. 




Long ago, people who found huge fossils did not know what they were. 
Some thought the big bones came from large animals that they had seen or 
read about, such as hippos or elephants. But some of the bones people found 
were too big to have come from even the biggest hippo or elephant. These 
enormous bones led some people to believe in giants. 



TIMSS Er PIRLS 

International Study Center 

Lynch School of Education, Boston College 


PIRLS 2016 SAMPLE PASSAGES, QUESTIONS, AND SCORING GUIDES 







Hundreds of years ago in France, a man named Bernard Palissy had 
another idea. He was a famous pottery maker. When he went to make his 
pots, he found many tiny fossils in the clay. He studied the fossils and wrote 
that they were the remains of living creatures. This was not a new idea. 

But Bernard Palissy also wrote that some of these creatures no longer lived 
on earth. They had completely disappeared. They were extinct. 

Was Bernard Palissy rewarded for his discovery? No! He was put in 
prison for his ideas. 

As time went by, some people became more open to new ideas about 
how the world might have been long ago. 

Then, in the 1820s, a huge fossil tooth was found in England. It is 
thought that Mary Ann Mantell, the wife of fossil expert Gideon Mantell 
was out for a walk when she saw what looked like a huge stone tooth. 

Mary Ann Mantell knew the big tooth was a fossil, and took it home to 
her husband. 



When Gideon Mantell first looked 
at the fossil tooth, he thought it had 
belonged to a plant eater because 
it was flat and had ridges. It was 
worn down from chewing food. It 
was almost as big as the tooth of an 
elephant. But it looked nothing like 
an elephant’s tooth. 


Fossil tooth sketched life-sized 




Gideon Mantell could tell that the pieces of rock attached to the tooth 
were very old. He knew that it was the kind of rock where reptile fossils 
were found. Could the tooth have belonged to a giant, plant-eating reptile 
that chewed its food? A type of reptile that no longer lived on earth? 


Gideon Mantell was really puzzled by the big tooth. No reptile that he 
knew about chewed its food. Reptiles gulped their food, and so their teeth 
didn’t become worn down. It was a mystery. 

Gideon Mantell took the tooth to a museum in London and showed it 
to other scientists. No one agreed with Gideon Mantell that it might be the 
tooth of a gigantic reptile. 

Gideon Mantell tried to find a reptile that had a tooth that looked like 
the giant tooth. For a long time, he found nothing. Then one day he met 
a scientist who was studying iguanas. An iguana is a large plant-eating 
reptile found in Central and South America. It can grow to be more than 
five feet long. The scientist showed Gideon Mantell an iguana tooth. At last! 
Here was the tooth of a living reptile that looked like the mystery tooth. 
Only the fossil tooth was much, much bigger. 


Iguana 


; A life-sized 
; drawing of an 
- J iguana's tooth 
from Gideon Mantelfs 
notebook 
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Now Gideon Mantell believed the fossil tooth had belonged to an animal 
that looked like an iguana. Only it wasn’t five feet long. Gideon Mantell 
believed it was a hundred feet long! He named his creature Iguanodon. 

That means “iguana tooth”. 

Gideon Mantell did not have a whole Iguanodon skeleton. But from 
the bones he had collected over the years, he tried to figure out what one 
might have looked like. He thought the bones showed that the creature had 
walked on all four legs. He thought a pointed bone was a horn. He drew an 
Iguanodon with a horn on its nose. 



What Gideon Mantell thought an Iguanodon looked like 
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Years later, several complete Iguanodon skeletons were found. They 
were only about thirty feet long. The bones showed that it walked on its 
hind legs some of the time. And what Gideon Mantell thought was a horn 
on its nose was really a spike on its “thumb”! Based on these discoveries, 
scientists changed their ideas about what the Iguanodon looked like. 



Gideon Mantell made some mistakes. But he had made 
an important discovery, too. Since his first idea that the 
fossil tooth belonged to a plant-eating reptile, he 
spent many years gathering facts and evidence 
to prove his ideas were right. By making 
careful guesses along the way, Gideon 
Mantell was one of the first 
people to show that long ago, 
giant reptiles lived on earth. 

And then they became extinct. 


Hundreds of years before, 
Bernard Palissy had been 
thrown in prison for saying 
nearly the same thing. But 
Gideon Mantell became 
famous. His discovery made 
people curious to find out more 
about these huge reptiles. 


What scientists today think the Iguanodon 
looked like 


In 1842, a scientist named Richard Owen decided that these extinct 
reptiles needed a name of their own. He called them Dinosauria. This 
means “fearfully great lizard”. Today we call them dinosaurs. 
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Questions The Giant Tooth Mystery 


1. What is a fossil? 

© the surface of rocks and cliffs 
® the bones of a giant 
© the remains of very old living things 
© the teeth of elephants 

2. According to the article, why did some people long ago believe in 
giants? 



3. Where did Bernard Palissy find fossils? 
© on the cliffs 
© in the clay 
© by a river 
© on a path 
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4. What was Bernard Palissy’s new idea? 



5. Why was Bernard Palissy put into prison? 

® People were not open to new ideas. 

(b) He copied his ideas from Gideon Mantell. 

(c) He left tiny fossils in his pottery. 

© Studying fossils was forbidden in France. 

6. Who found the fossil tooth in England? 

® Bernard Palissy 

© Mary Ann Mantell 
© Richard Owen 
© Gideon Mantell 


©I 
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7. What did Gideon Mantell know about reptiles that made the fossil 
tooth puzzling? 

® Reptiles had no teeth. 

® Reptiles were found under rocks. 

© Reptiles lived long ago. 

© Reptiles gulped their food. 


8. Gideon Mantell thought the tooth might have belonged to different 
types of animals. Complete the table to show what made him think 
this. 


Type of animal 

What made him think this 

A plant eater 

The tooth was flat with ridges. 

A giant creature 


A reptile 
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9. Why did Gideon Mantell take the tooth to a museum? 
® to ask if the fossil belonged to the museum 
© to prove that he was a fossil expert 
© to hear what scientists thought of his idea 
© to compare the tooth with others in the museum 


10. A scientist showed Gideon Mantell an iguana tooth. Why was this 
important to Gideon Mantell? 




11. What did Gideon Mantell use when trying to figure out what the 
Iguanodort looked like? 

© bones he collected 

© ideas from other scientists 

© pictures in books 

© teeth from other reptiles 
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12. Look at the two pictures of the Iguanodon. What do they help you 
to understand? 




13. Later discoveries proved that Gideon Mantell was wrong about 

what the Iguanodon looked like. Fill in the blanks to complete the 
table. 


What Gideon Mantell thought 
the Iguanodon looked like 

What scientists today think 
the Iguanodon looked like 

The Iguanodon walked on four legs. 



The Iguanodon had a spike on 
its thumb. 

The Iguanodon was 100 feet long. 





© 

© 


© 

© 


©1 

© 

© 
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© 

© 


© 

© 
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14. What were found that showed Gideon was wrong about what the 
Iguanodon looked like? 

® more fossil teeth 

® scientific drawings 

© living Iguanodons 

© whole skeletons 
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Giant Tooth Mystery, Item 2 

2. According to the article, why did some people long ago believe in giants? 

Process: Make Straightforward Inferences 

1 - Acceptable Response 

The response demonstrates understanding that people long ago believed in giants because they found huge 
bones/skeletons/fossils. 

NOTE TO SCORERS: Some students use the word ‘giant’ as a synonym for ‘big’ or ‘huge’. Such 
responses should be credited only where the meaning is made clear. 

Examples: 

They found bones too big to belong to something they knew. 

They found giant bones that were too big to be from the biggest hippo. 

They found really big bones. 

The bones were so big they must be from giants. 

0 - Unacceptable Response 

The response does not demonstrate understanding that people long ago believe in giants because they 
found huge bones/skeletons/fossils. 

Examples: 

Giants are really big. 

They found giant bones. (Please note that the use of giant’ is ambiguous.) 

They found things that must belong to giants. 

They found dinosaur bones. 

They found bones from giants. 


Giant Tooth Mystery, Item 4 

4. What was Bernard Palissy’s new idea? 

Process: Interpret and Integrate Ideas and Information 

1 - Acceptable Response 

The response demonstrates understanding that Palissy’s new idea was that some fossils belonged to animals 
that no longer lived on earth, had completely disappeared, or were extinct. 

Examples: 

Fossils could be from extinct animals. 

Some belonged to creatures no longer living on earth. 

His idea was that some animals had completely disappeared! 

0 - Unacceptable Response 

The response does not demonstrate understanding of Palissy’s new idea. It might relate to Palissy’s idea that 
fossils once belonged to living creatures, or may state a fact about Palissy’s work. 

Examples: 

Fossils were from the remains of living creatures. 
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Reptiles were extinct. 

He found fossils in his clay. 

He was a famous pottery maker. 
He studied fossils. 


Giant Tooth Mystery, Item 8 

8. Gideon Mantell thought the tooth might have belonged to different types of 
animals. Complete the table to show what made him think this. 


Type of animal 

What made him think this 

A plant eater 

The tooth was flat with ridges 

A giant creature 


A reptile 



Process: Interpret and Integrate Ideas and Information 

NOTE TO SCORERS: Each of the two parts of this item will be scored separately in its own 
I -point coding block. 

The entire item, with acceptable responses for each of the two parts and the corresponding coding blocks, 
should look like this: 


Type of animal 

What made him think this 

A plant eater 

The tooth was flat with ridges 

A giant creature 

The response identifies the large size of the fossil 
tooth (as big as an elephant’s tooth) 

A reptile 

The response indicates that: 

the rock in which it was found was the kind of 
rock where reptile fossils were found/it was found 
where reptiles had lived, OR 

the fossil tooth was similar to/looked like an 
iguana/reptile tooth 


© 

> ® 

© 


© 

© 


* © 

© 



TIMSS Er PIRLS 

International Study Center 

Lynch School of Education, Boston College 


PIRLS 2016 SAMPLE PASSAGES, QUESTIONS, AND SCORING GUIDES 


131 


PIRLS 2016 FRAMEWORK: / CO ’ in < z cl _j in cl m qC -i in cl < in in < u lu in 




PIRLS 


2016 


A GIANT CREATURE 
1 - Acceptable Response 

The response shows understanding of the characteristics that indicate the fossil tooth could belong to a 
giant creature. 


Type of animal 

What made him think this 

A plant eater 

The tooth was flat with ridges 

A giant creature 

The response identifies the large size of the fossil 
tooth (as big as an elephant’s tooth) 

A reptile 

The response indicates that: 

the rock in which it was found was the kind of 

rock where reptile fossils were found/it was found 

where reptiles had lived, OR 

the fossil tooth was similar to/looked like an 

iguana/reptile tooth 


0 - Unacceptable Response 


The response does not show understanding of the characteristics that indicate the fossil tooth could belong 
to a giant creature. The response may refer to the text at the beginning of the passage about fossils in 
general, rather than to Gideons hypotheses about the fossil tooth. 

Examples: 

Some thought the big bones came from large animals. 

It was worn down. 

It looked like an elephant’s tooth. (Please note that this is an inaccurate response. The text states, “it looked 
nothing like an elephant’s tooth.”) 

A REPTILE 

1 - Acceptable Response 

The response shows understanding of the characteristics that indicate the fossil tooth could belong to a 
reptile. 


Type of animal 

What made him think this 

A plant eater 

The tooth was flat with ridges 

A giant creature 

The response identifies the large size of the fossil 
tooth (as big as an elephant’s tooth) 

A reptile 

The response indicates that: 

1) the rock in which it was found was the kind 
of rock where reptile fossils were found/it was 
found where reptiles had lived, OR 

1) the fossil tooth was similar to/looked like an 
iguana/reptile tooth 
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0 - Unacceptable Response 

The response does not show understanding of the characteristics that indicate the fossil tooth could belong 
to a reptile. 

Examples: 

It eats plants. 

Reptiles gulped their food. 


Giant Tooth Mystery, Item 10 

1 0. A scientist showed Gideon Mantel! an iguana tooth. Why was this important to 
Gideon Mantell? 

Process: Interpret and Integrate Ideas and Information 

1 - Acceptable Response 

The response demonstrates understanding that the iguana tooth provided evidence that supported Gideon 
Mantell’s theory that the fossil tooth might have belonged to a giant reptile. 

Examples: 

The iguana tooth showed his fossil could be from a reptile. 

It helped him find out what type of animal the tooth belonged to. 

The tooth proved he was right. 

It gave him proof for what he thought all along. 

Or, the response demonstrates a more general understanding that the iguana tooth looked like the fossil 
tooth. 

Examples: 

The iguana tooth looked like the fossil tooth. 

He could see that they looked the same. 

He could tell it was the same one. 

He had spent years looking for a matching tooth. 

It was flat and had ridges. 

0 - Unacceptable Response 

The response does not demonstrate understanding of the significance of the iguana tooth. 

Examples: 

He wanted to be famous. 

He thought it would be interesting to see an iguana’s tooth. 

He wanted to learn more about reptiles. 

It showed he was clever. (Please note that this response is too vague as it focuses on his personal 
characteristics rather than his discovery.) 

He wanted to compare the teeth. (Please note that this response fails to indicate the significance of the 
comparison.) 
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Giant Tooth Mystery, Item 12 

1 2. Look at the two pictures of the Iguanodon. What do they help you to 
understand? 

Process: Examine and Evaluate Content, Language, and Textual Elements 

2 - Complete Comprehension 

The response demonstrates understanding that the pictures show the changes in scientific ideas, or that the 
pictures show different people’s ideas about the Iguanodon. 

Examples: 

that scientists today think the Iguanodon looked different than Gideon Mantell did 
To show how people’s ideas about what the Iguanodon looked like changed. 

To show that different people had different ideas about what it looked like, 
how different the ideas were 

Gideon Mantell thought the bones showed the Iguanodon walked on all four legs, but later scientists 
changed their minds. 

Or, the response indicates that the pictures illustrate the mistakes that Gideon Mantell or other people 
might have made. 

Examples: 

To show that Gideon got some things wrong, 
that people sometimes make mistakes 

1 - Partial Comprehension 

The response demonstrates a more general understanding that the Iguanodons looked different in the two 
pictures. 

Example: 

To show they look different. 

Or, the response describes a difference between the two pictures without reference to changes in scientific 
ideas or what different people might have believed. 

Example: 

One has 4 legs, the other has 2. 

Or, the response provides an explicit reference to one of the pictures without reference to changes in 
scientific ideas or what different people might have believed. 

Example: 

That Gideon thought it had a horn. 
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0 - No Comprehension 


The response does not demonstrate understanding of the purpose of the illustrations. The response may 
describe a specific feature from one of the pictures, or give a description of what the illustrations have in 
common. 


Or, the response may provide an inaccurate interpretation that the Iguanodon itself changed in appearance 
over time, rather than peoples ideas. 

Examples: 

To show what they looked like. 

They help you understand how the Iguanodon changed over the years. 

They show me they ate plants. 

They had 4 legs. 


Giant Tooth Mystery, Item 13 

1 3. Later discoveries proved that Gideon Mantell was wrong about what the 
Iguanodon looked like. Fill in the blanks to complete the table. 


What Gideon Mantell thought the 
Iguanodon looked like 

What scientists today think the 
Iguanodon looked like 

The Iguanodon walked on four legs 



The Iguanodon had a spike on its 
thumb 

The Iguanodon was 100 feet long 



Process: Interpret and Integrate Ideas and Information 


NOTE TO SCORERS: Each of the three parts of this item will be scored separately in its own 
I -point coding block. 

The entire item, with acceptable responses for each of the three parts and the corresponding coding blocks, 
should look like this: 


What Gideon Mantell thought the 
Iguanodon looked like 

What scientists today think the 
Iguanodon looked like 

The Iguanodon walked on four legs 

The Iguanodon (sometimes) walked/ 
stood on two/hind legs 

The Iguanodon had a horn (on its head/ 
face/nose) 

OR, the spike was on its head/face/nose 

The Iguanodon had a spike on its 
thumb 

The Iguanodon was 100 feet long 

The Iguanodon was 30 feet (9 metres) 
long 


0 

® 


0 

* 0 


© 

0 

© 


> 
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1 - Acceptable Response 

The response shows understanding of the difference in the way that Gideon Mantell and scientists today 
think the Iguanodon walked or stood. 


What Gideon Mantell thought the 
Iguanodon looked like 

What scientists today think the 
Iguanodon looked like 

The Iguanodon walked on four legs 

The Iguanodon (sometimes) walked/ 
stood on two/hind legs 

The Iguanodon had a horn (on its 
head/face/nose) 

OR, the spike was on its head/face/nose 

The Iguanodon had a spike on its 
thumb 

The Iguanodon was 100 feet long 

The Iguanodon was 30 feet (9 metres) 
long 


0 - Unacceptable Response 

The response does not show understanding of the way scientists today think the Iguanodon walked or 
stood. 

Examples: 

two 

It stood. 

1 - Acceptable Response 

The response shows understanding of the difference in where Gideon Mantell and scientists today think the 
Iguanodon had a spike. 


What Gideon Mantell thought the 
Iguanodon looked like 

What scientists today think the 
Iguanodon looked like 

The Iguanodon walked on four legs 

The Iguanodon (sometimes) walked/ 
stood on two/hind legs 

The Iguanodon had a horn (on its 
head/face/nose) 

OR, the spike was on its head/face/nose 

The Iguanodon had a spike on its 
thumb 

The Iguanodon was 100 feet long 

The Iguanodon was 30 feet (9 metres) 
long 
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0 - Unacceptable Response 


The response does not show understanding of where Gideon Mantell thought the Iguanodon had a spike. 
Examples: 
horn on its thumb 
spike on its back 

did not have a spike on its thumb 

1 - Acceptable Response 

The response shows understanding of the difference in what Mantell and scientists today think was the 
length of the Iguanodon. 


What Gideon Mantell thought the 
Iguanodon looked like 

What scientists today think the 
Iguanodon looked like 

The Iguanodon walked on four legs 

The Iguanodon (sometimes) walked/ 
stood on two/hind legs 

The Iguanodon had a horn (on its 
head/face/nose) 

OR, the spike was on its head/face/nose 

The Iguanodon had a spike on its 
thumb 

The Iguanodon was 100 feet long 

The Iguanodon was 30 feet (9 metres) 
long 


0 - Unacceptable Response 

The response does not show understanding of how long scientists today think the Iguanodon was. 
Examples: 

It was not 100 feet long. 

5 feet long 
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Questions, and Scoring Guides 


Reading for Literary Experience 
Brave Charlotte 


Reading to Acquire and Use Information 
Caterpillar to Butterfly 
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Brave Charlotte 

By Anu Stohner 
Illustrated by Henrike Wilson 

Charlotte was different from 
all the other sheep right from the 
start. When all the other lambs 
just stood shyly by their mothers, 
Charlotte was leaping around, 
ready for adventure. 

Charlotte lived with all the other 
sheep on a hillside far from the 
farm. They had a shepherd to look 
after them and he had an old dog 
named Jack. Jack tried to keep 
Charlotte under control, but she 
wasn’t scared of him. 


•f If// 1 !! / 
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1. Who is Jack? 



©1 

®\ 

© 
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2. What did Jack try to do with Charlotte? 



©1 

© 

© 
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One time, Charlotte jumped over the side of a riverbank 
and went for a swim in the fast-running stream. 

“Tut, tut,” said the older sheep, shaking their heads. 

What would they say if they knew that at night Charlotte 
secretly roamed through the countryside? 

When all the other sheep 
were sleeping, she would 
slip away to her special 
place and gaze at the moon. 
Even Jack didn’t notice. 

But he didn’t have very 
good ears these days. 
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3. Give two ways that Charlotte was different from the 
other sheep. 




2 . 


4. Where was Charlotte standing in the picture on the 
opposite page? 

© in her special place 

® by the stream 

© at the farm 

© in her hiding place 


5. Why didn’t Jack notice when Charlotte went out at 
night? 
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One day something terrible happened. The shepherd fell 
over and broke his leg. Jack barked and circled around 
him, but that didn’t help 
one bit. The shepherd lay 
in the grass, not knowing 
what to do. 

“Oh dear, oh dear,” 
said the older sheep. 

“Somebody must go to 
the farmer’s house in the 
valley and get help.” 

“Jack should go. He is 
the only one who knows the way.” 

“But it is too far. He hardly manages with the herd these 
days.” 

“Yes, that’s true,” said the others, shaking their heads in 
despair. 
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6. Why was the shepherd lying in the grass? 
© He didn’t want to visit the farmer. 

© He had broken his leg. 

© He wanted to go to sleep. 

© He was gazing at the moon. 


7. Where did the animals need to go to get help? 



©1 
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8. Give one reason why Jack was the best one to get 
help. 



Give one reason why Jack was NOT the best one to 
get help. 


© 

® 
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Then Charlotte said, “I’ll do it. I’ll go.” 

“Charlotte?” muttered the older sheep. 

“Out of the question! A sheep has never gone to the valley 
alone.” 

The older sheep were beside themselves with worry. But 
Charlotte couldn’t hear them. She had already left to find 
the right way to the valley. 

She bounded over fields, through the stream, and over 
the mountain. 
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9. How did the older sheep feel when Charlotte left? 
© happy 
© relieved 
© worried 
© angry 


10. Why couldn’t Charlotte hear the older sheep? 



©1 
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When Charlotte reached the busy road, it was the middle 
of the night. She stood and watched the traffic. 

A truck driver noticed Charlotte and stopped in the road. 

“Going to the valley?” he asked. Charlotte nodded. 
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11. When did Charlotte reach the road? 



©1 

®\ 

© 

®j 


12. What did Charlotte do when she reached the road? 
© She walked towards the farmer’s house. 

® She stood and watched the traffic. 

© She tried to cross the road. 

© She waved for a truck to stop. 
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It was so nice to speed along in the truck that Charlotte 
was almost sorry when they reached the farmer’s house. 

The farmer was 
asleep when Charlotte 
tapped on his window 
with her nose. 

“It’s Charlotte,” 
said the farmer, 

“and she is all alone. 

Something must be 
wrong.” 
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13. Why was Charlotte almost sorry to reach the 
farmer’s house? 

© She missed the other sheep. 

© She would have to wake up the farmer. 

© She liked the ride in the truck. 

© She was worried about the shepherd. 


14. How did the farmer know something was wrong? 
© Charlotte was all alone. 

® Charlotte told him. 

© Charlotte came in a truck. 

© Charlotte looked sorry. 
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Charlotte and the farmer drove on the tractor to find the 
other sheep. When they arrived, the poor shepherd was 
still lying in the grass. The farmer took him to the hospital 
right away. 

The shepherd had his leg in a cast for six weeks before 
he could go back to the sheep. When he returned, he gave 
Charlotte a big smile. From then on, Charlotte was free to 
roam as she pleased. 
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15. What did the farmer and Charlotte do together? 




16. How long did the shepherd have a cast on his leg? 




17. Why did the shepherd give Charlotte a big smile? 
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18. Put the events from the story in the correct order. 
The first one has been done for you. 



The shepherd goes to the hospital. 

Charlotte goes to the valley. 

1 The shepherd needs help. 

The farmer finds the shepherd. 
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Scoring Guides for Constructed-response Questions 


Brave Charlotte, Item 1 

1 . Who is Jack? 

Purpose: Literary 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 


The response indicates that Jack is an (old sheep) dog. 

Examples: 
old sheep dog 
sheep dog 
dog 

shepherd’s dog 

0 - Unacceptable Response 

The response does not indicate that Jack is an old sheep dog. The response may provide an overly vague 
description of Jack or describe something that Jack does in the story. 

Examples: 

sheep 

shepherd 

animal (Please note that this response is too vague.) 

He’s the one that takes care of the sheep. 

He keeps Charlotte under control. 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Brave Charlotte, Item 2 

2. What did Jack try to do with Charlotte? 

Purpose: Literary 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 

The response indicates that Jack tries to keep Charlotte under control. Please note that responses 
paraphrasing this idea are considered acceptable. 

Examples: 

keep Charlotte under control 

control 

keep her calm 

0 - Unacceptable Response 

The response does not indicate that Jack tries to keep Charlotte under control. 

Examples: 

put Charlotte to sleep 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Brave Charlotte, Item 3 

3. Give two ways that Charlotte was different from the other sheep. 

Purpose: Literary 

Process: Interpret and Integrate Ideas and Information 


2 - Complete Comprehension 

The response provides two text-based ways that Charlotte was different from the other sheep from the list 
of acceptable responses below. Ways listed on the same line are considered the same idea and cannot be 
credited separately. 

NOTE TO SCORERS: Responses paraphrasing these ideas are considered acceptable as long as 
the meaning is clear. 

Ways that Charlotte was different from other sheep: 

She was brave. / She was not shy. 

She was leaping around (while other sheep stood by their mothers). / She didn’t stand by her mother. 

She was ready for adventure. 

She jumped over a riverbank and went swimming. 

She secretly roamed the countryside at night. 

She slipped away to her special place (and gazed at the moon) (while other sheep were sleeping). 

She went to save the shepherd. (Please note that specific actions that were involved in saving the 
shepherd, such as riding in a truck, are also acceptable.) 

1 - Partial Comprehension 

The response provides one text-based way that Charlotte was different from the other sheep from the list of 
acceptable responses above. 

0 - No Comprehension 

The response provides does not provide any text-based way that Charlotte was different from the other 
sheep. 

Examples: 

She wasn’t scared of Jack. (Please note that the text does not state that the other sheep are scared of Jack.) 
She was shy. 

She was scared. 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Brave Charlotte, Item 5 

5. Why didn’t Jack notice when Charlotte went out at night? 

Purpose: Literary 

Process: Make Straightforward Inferences 

1 - Acceptable Response 

The response indicates that Jack did not notice Charlotte because he could not hear very well. 

Examples: 

He didn’t have very good hearing. 

His ears were bad. 

He didn’t hear her. 

0 - Unacceptable Response 

The response does not recognize that Jack could not hear very well. The response may indicate that Jack was 
old, but does not connect this to his lack of hearing. 

Examples: 

He was asleep. 

He was old. (Please note that this response is too vague to be considered accurate.) 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Brave Charlotte, Item 7 

7. Where did the animals need to go to get help? 

Purpose: Literary 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 

The response identifies the farmer’s house in the valley as the place to go for help. 

Examples: 
the farmer’s house 
to the valley 
to the farmer 

0 - Unacceptable Response 

The response does not identify the farmer’s house in the valley as the place to go for help. 

Examples: 
a farm 

to the road (Please note that this is an intermediate location, not the destination.) 
to the stream 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 
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Brave Charlotte, Item 8 

8. Give one reason why Jack was the best one to get help. Give one reason why 
Jack was NOT the best one to get help. 

Purpose: Literary 

Process: Make Straightforward Inferences 

2 - Complete Comprehension 

The response provides a plausible text-based reason why Jack was the best one to get help AND a reason 
why Jack was not the best one to get help from the lists below. 

Why Jack was the best: 

He knew where to go. 

He knew how to get to the farmer’s house/valley. 

Why Jack was NOT the best: 

He is too old to go that far. 

It was too far for him to travel. 

He can’t manage the distance. 

He needed to stay and watch the sheep. 

He’s too slow/weak. 

He hardly manages with the herd these days. 

1 - Partial Comprehension 

The response provides a plausible text-based reason why Jack was the best one to get help OR a reason why 
Jack was not the best one to get help from the lists above. 

0 - No Comprehension 

The response does not provide a plausible text-based reason for why Jack was the best one to get help or a 
reason why he was not the best one to get help. 

Example: 

He was the best because he was the fastest. 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Brave Charlotte, Item 10 

1 0. Why couldn’t Charlotte hear the older sheep? 

Purpose: Literary 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 

The response indicates that Charlotte could not hear the sheep because she had already left to go to the 
farmers house. 

Examples: 

She had already left. 

She already was on her way to the farmer’s house. 

She was too far away. 

0 - Unacceptable Response 

The response does not indicate that Charlotte had already left to go to the farmer’s house. 

Example: 

She had bad ears. 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 



TIMSS Er PIRLS 

International Study Center 

Lynch School of Education, Boston College 


prePIRLS 2016 SAMPLE PASSAGES, QUESTIONS, AND SCORING GUIDES 161 


PIRLS 2016 FRAMEWORK: / (J \ in < z a- _ i in cl i_ <d cl i- cc Jin cl < in in < u lu in 



PIRLS 


2016 


Brave Charlotte, Item 1 1 

1 1 . When did Charlotte reach the road? 

Purpose: Literary 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 

The response identifies the time (middle of the night) that Charlotte reached the road. 

Examples: 

in the middle of the night 

at night 

night 

0 -Unacceptable Response 

The response does not identify the time that Charlotte reached the road. 

Examples: 
in the middle 

when the farmer was sleeping (Please note that the text does not state that the farmer was asleep when 
Charlotte reached the road.) 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Brave Charlotte, Item 15 

1 5. What did the farmer and Charlotte do together? 

Purpose: Literary 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 

The response indicates that Charlotte and the farmer drove a tractor out to the sheep. 

Examples: 

drove a tractor out to the sheep 
drove 

They rode together. 

found the sheep 

They went to get the shepherd. 

0 - Unacceptable Response 

The response does not indicate that Charlotte and the farmer drove a tractor out to the sheep. The 
response may provide an action from the wrong part of the text. 

Examples: 

They went to the hospital. (Please note that the farmer did this with the shepherd.) 

They went to the valley. (Please note that Charlotte did this on her own.) 

They rode in a truck. (Please note that Charlotte did this earlier in the story.) 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Brave Charlotte, Item 16 

1 6. How long did the shepherd have a cast on his leg? 

Purpose: Literary 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 

The response indicates that the shepherd wore the cast for six (6) weeks. 

Example: 
six weeks 
6 weeks 

0 - Unacceptable Response 

The response does not indicate that the shepherd wore the cast for six weeks. 

Example: 

six 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Brave Charlotte, Item 1 7 

1 7. Why did the shepherd give Charlotte a big smile? 

Purpose: Literary 

Process: Interpret and Integrate Ideas and Information 

1 - Acceptable Response 

The response indicates that the shepherd smiled at Charlotte because he was grateful to her (for helping 
him/getting the farmer), or the response may provide a more general indication that the shepherd was 
proud of Charlotte. 

Examples: 

She had gone to get the farmer. 

She had gone to get help. 

He was thankful. 

She saved him. 

She helped him. 

He was proud of her. 
because she was brave 

0 - Unacceptable Response 

The response does not indicate that the shepherd smiled at Charlotte because she had helped him 
by getting the farmer or because he was proud of her. 

Examples: 

He liked her. 

He was happy. 

From then on, Charlotte was free to roam as she pleased. 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Brave Charlotte, Item 18 

1 8. Put the events from the story in the correct order. The first one has been done 
for you. 

The shepherd goes to the hospital. 

Charlotte goes to the valley. 

1_ The shepherd needs help. 

The farmer finds the shepherd. 

Purpose: Literary 

Process: Interpret and Integrate Ideas and Information 

1 - Acceptable Response 

The response recognizes the order of the events from the story (4, 2, 1, 3). 

4 The shepherd goes to the hospital. 

2 Charlotte goes to the valley. 

1_ The shepherd needs help. 

3 The farmer finds the shepherd. 

0 - Unacceptable Response 

The response does not recognize the order of the events from the story. 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Caterpillar to Butterfly 

By Deborah Heiligman 
Illustrated by Bari Weissman 


One day our teacher brought a caterpillar to school in a 
jar. It was eating green leaves. This tiny caterpillar was 
going to change into a butterfly. Caterpillars usually turn 
into butterflies outdoors. But we watched our caterpillar 
change into a butterfly right in our classroom. 
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1. What was the caterpillar eating when the teacher 
first showed it to the children? 



©1 

®\ 

© 

®j 


2. Where did the children watch the caterpillar turn 
into a butterfly? 



©] 

© 

© 

®| 
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Our teacher told us our caterpillar started out as a 
tiny egg. The mother butterfly laid the egg on a leaf. 

The mother butterfly chose the leaf of a plant that the 
caterpillar would eat. When the caterpillar hatched out 
of the egg, it was hungry. It ate its own eggshell! Then it 
started to eat green plants right away. 

The caterpillar’s job was to eat and eat, so it would grow. 
Each day it got bigger. The caterpillar ate and grew for 12 
days. 
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3. Why did the egg need to be on a leaf? 



©1 

®\ 

© 

®j 


4. What was the first thing the caterpillar did when it 
hatched? 

© chose a leaf 

® ate its eggshell 

© ate green plants 

© laid an egg 
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Our skin grows with us. But a caterpillar’s skin does not 
grow. When the caterpillar got too big for its skin, the skin 
split down the back. 



The caterpillar crawled right out of its own skin. It 
had a new skin underneath. This is called shedding. Our 
caterpillar shed its skin four times. After many days our 
caterpillar was finished growing. It was almost as big as 
my little finger. 
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5. How is a caterpillar’s skin different from human 
skin? 

© It does not grow. 

® It is too big for the caterpillar. 

© It grows for many days. 

© A caterpillar can crawl in its skin. 

6. Why did the caterpillar need to shed its skin? 

© Its skin got smaller. 

® It was hungry. 

© It got too big for its skin. 

© Its skin got too old. 

7. How many times did the caterpillar shed its skin? 



8. How big was the caterpillar when it finished 
growing? 



®1 

® 

® 

®j 
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Then our caterpillar made a special house. First it made 
a button of silk. It used this button to hang upside down 
from a twig. 


Button of silk 



Then it shed its skin for the last time. Instead of a new 
skin, this time it formed a chrysalis with a hard shell. We 
watched the chrysalis for a long time. 


Chrysalis with a 
hard shell 


Every day the shell looked 
the same. But inside the shell 
changes were happening. 


7 €aterfrfffei°t6 ( Butterfly 






9. What did the caterpillar use a button of silk for? 



©1 

® 

® 

®| 


10. Which word tells you something was happening to 
the chrysalis inside its shell? 

© changes 

® button 

© shed 

© watched 
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We waited and waited. Then, one day, somebody shouted, 
“Look!” The shell was cracking. It was a butterfly! Our 
butterfly was damp and crumpled. It hung on to the shell 
while its wings flapped. This pumped blood into its wings. 
The wings stretched out and dried. Soon our butterfly was 
ready to fly. 
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11. What came out of the shell? 
© a green leaf 
® a new caterpillar 
© a tiny egg 
© a butterfly 


12. How did the butterfly pump blood into its wings? 
© by drying them 
® by flapping them 
© by stretching them 
© by crumpling them 
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Our butterfly could not stay in the jar. It needed to be 
outside with flowers and grass and trees. We watched 
our butterfly land on a flower. It sipped the flower’s 
nectar through a long, coiled tube. Maybe it was a female 
butterfly. Maybe someday she would lay an egg on a leaf. 
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13 . 


14 . 


Why couldn’t the butterfly stay in the jar? 
© They did not want to watch it anymore. 
© They wanted another caterpillar. 

© It was too big for the jar. 

© It needed to be free outdoors. 


How did the butterfly get the flower nectar? 
© ate flowers and grass 
® chewed green leaves 
© sipped through a tube 
© pumped its wings 
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15. Put what happens to a caterpillar as it changes into 
a butterfly in the correct order. The first one has 
been done for you. 



The caterpillar forms a hard shell. 

1 The caterpillar eats and grows. 

The butterfly flaps its wings. 

The shell of the chrysalis cracks. 



16. Think about the whole article. Why do you think the 
teacher brought the caterpillar into the classroom? 
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Scoring Guides for Constructed-response Questions 


Caterpillar to Butterfly, Item 1 

1 . What was the caterpillar eating when the teacher first showed it to the children? 

Purpose: Acquire and Use Information 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 

The response indicates that the caterpillar was eating leaves. 

Examples: 

The caterpillar was eating green leaves. 

leaves 

plants 

0 - Unacceptable Response 

The response does not indicate that the caterpillar was eating leaves. 

Example: 

food 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Caterpillar to Butterfly, Item 2 

2. Where did the children watch the caterpillar turn into a butterfly? 

Purpose: Acquire and Use Information 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 

The response indicates that the children watched the caterpillar change in a jar or, more generally, in their 

classroom/at school. 

Examples: 
in ajar 
in the class 
at school 

0 - Unacceptable Response 

The response does not indicate that the children watched the caterpillar change in ajar or in their classroom. 
Examples: 
outside 
on a leaf 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 

control. 

9 - Blank 
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Caterpillar to Butterfly, Item 3 

3. Why did the egg need to be on a leaf? 

Purpose: Acquire and Use Information 

Process: Interpret and Integrate Ideas and Information 

1 - Acceptable Response 

The response indicates that the egg was laid on a green leaf so that the caterpillar would have food to eat (as 
soon as it hatched). 

Examples: 

so it would have food 
The caterpillar could eat it. 

The caterpillar would be hungry, 
to eat it 

0 - Unacceptable Response 

The response does not indicate that the egg was laid on a leaf so that the caterpillar would have food to eat 
when it hatched. 

Examples: 

so the butterfly could eat it 
so it could grow 

to eat (Please note that this response is too vague.) 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Caterpillar to Butterfly, Item 7 

7. How many times did the caterpillar shed its skin? 

Purpose: Acquire and Use Information 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 

The response indicates that the caterpillar sheds its skin four (4) times. 

Examples: 

four times 
four- 4 

0 - Unacceptable Response 

The response does not recognize that the caterpillar sheds four times. 

Examples: 

four days 
five times 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Caterpillar to Butterfly, Item 8 

8. How big was the caterpillar when it finished growing? 

Purpose: Acquire and Use Information 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 

The response indicates that the caterpillar was the size of a little finger when it finished growing. 
Examples: 

It was as big as my little finger, 
the size of a boy’s little finger 
finger 

0 - Unacceptable Response 

The response does not indicate that the caterpillar was the size of a little finger when it finished growing. 
Examples: 
as big as a hand 
big 

It was small. 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Caterpillar to Butterfly, Item 9 

9. What did the caterpillar use a button of silk for? 

Purpose: Acquire and Use Information 

Process: Focus on and Retrieve Explicitly Stated Information 

1 - Acceptable Response 

The response recognizes that the caterpillar uses a button of silk to hang (from a twig). 

Examples: 

to hang upside down 
to hang 

hang from a twig 

0 - Unacceptable Response 

The response does not recognize that the caterpillar uses a button of silk to hang from a twig. The response 
may repeat words from the question. 

Examples: 
to eat 

to make a house 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Caterpillar to Butterfly, Item 15 


1 5. Put what happens to a caterpillar as it changes into a butterfly in the correct 
order. The first one has been done for you. 

The caterpillar forms a hard shell. 

1_ The caterpillar eats and grows. 

The butterfly flaps its wings. 

The shell of the chrysalis cracks. 

Purpose: Acquire and Use Information 

Process: Interpret and Integrate Information and Ideas 

1 - Acceptable Response 

The response provides the correct order of the events in the article (2, 1, 4, 3). 

2 The caterpillar forms a hard shell. 

1_ The caterpillar eats and grows. 

4 The butterfly flaps its wings. 

3 The shell of the chrysalis cracks. 

0 - Unacceptable Response 

The response does not provide the correct order of the events in the article. 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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Caterpillar to Butterfly, Item 16 

1 6. Think about the whole article. Why do you think the teacher brought the 
caterpillar into the classroom? 

Purpose: Acquire and Use Information 

Process: Examine and Evaluate Content, Language, and Textual Elements 

1 - Acceptable Response 

The response recognizes that the teacher brought the caterpillar in to class for students to see it change/ 
grow (into a butterfly). 

Examples: 

to see it change into a butterfly 
to learn how butterflies form 
so they could see it change 
because she wanted them to see it grow 

OR, the response may recognize that the teacher wanted students to learn about butterflies, without 
explicitly mentioning change from a caterpillar. 

Examples: 

to learn about butterflies 
to show how a butterfly works 

OR, the response may indicate a general understanding that the teacher wanted students to learn about the 
caterpillar’s cycle of life without explicitly mentioning its change into a butterfly 
Examples: 

to learn about the cycle of life 

to learn about the caterpillar’s cycle 

to learn how a caterpillar’s life begins and ends 

0 - Unacceptable Response 

The response provides only a vague understanding of the reason the teacher brought the caterpillar into 
class, or may provide an inaccurate description of why the teacher brought in the caterpillar. 

Examples: 

to learn about caterpillars 
to watch it eat 
to learn about nature 
It was neat. 

Non-Response Codes 

8 - Not administered. Question misprinted, page missing, or other reason out of student’s 
control. 

9 - Blank 
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